00010000:00010000:2.0F:1761225262.448996:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1761225263.473015:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0F:1761225263.923995:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225263.924008:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225263.924012:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225263.924025:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 857 00000100:00100000:0.0F:1761225263.924027:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720587072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761225263.924031:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 11s 00000100:00080000:1.0:1761225263.924041:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225263.924053:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225263.924072:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720587200/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225263.924086:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225263.924090:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225263.924108:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720587008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225263.924140:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720587136:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225263.932624:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720587072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225263.932751:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720587008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225263.932765:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720587136:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0F:1761225263.933435:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720587200:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761225263.969978:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225263.969994:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225263.970006:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720587200:192.168.202.114@tcp:8:kworker/u8:0.0 00010000:00010000:1.0:1761225263.970074:0:2285:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 6/2,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970091:0:2285:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (FAILED) ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 6/2,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970109:0:2285:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 6/2,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970123:0:2285:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 6/2,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970129:0:2285:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 5/1,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970157:0:2285:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 4/1,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970164:0:2285:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 3/0,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970172:0:2285:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 4/0,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970179:0:2285:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000ad7b8b45 already unused, calling callback (000000007e95e7ea) 00010000:00010000:1.0:1761225263.970183:0:2285:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 5/0,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970195:0:2285:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1761225263.970200:0:2285:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 3/0,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1761225263.970206:0:2285:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc1d lrc: 0/0,0 mode: --/PR res: [0x3b:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 20347 timeout: 0 lvb_type: 1 00000100:00100000:1.0:1761225263.970240:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:0:1846777720586944:192.168.202.114@tcp:101:stat.0 00000020:00010000:2.0:1761225263.971355:0:20347:0:(cl_lock.c:53:__cl_lock_trace()) release lock: 000000001f7728bb (000000004616047a/0) at cl_lock_release():234 00000020:00010000:2.0:1761225263.971368:0:20347:0:(cl_lock.c:53:__cl_lock_trace()) cancel lock: 000000001f7728bb (000000004616047a/0) at cl_lock_cancel():151 00000020:00010000:2.0:1761225263.971376:0:20347:0:(cl_lock.c:53:__cl_lock_trace()) cancel lock: 000000004d658d91 (00000000c4a316f2/1) at cl_lock_cancel():151 00000020:00010000:2.0:1761225263.971381:0:20347:0:(cl_lock.c:53:__cl_lock_trace()) destroy lock: 000000001f7728bb (000000004616047a/0) at cl_lock_fini():84 00000020:00010000:2.0:1761225263.971384:0:20347:0:(cl_lock.c:53:__cl_lock_trace()) destroy lock: 000000004d658d91 (000000004616047a/1) at cl_lock_fini():84 00000080:00200000:2.0:1761225263.971405:0:20347:0:(vvp_io.c:313:vvp_io_fini()) [0x200000402:0x30:0x0] ignore/verify layout 0/1, layout version 2 need write layout 0, restore needed 0 00000080:00200000:2.0:1761225263.971414:0:20347:0:(vvp_io.c:1820:vvp_io_init()) [0x200000402:0x30:0x0] ignore/verify layout 0/1, layout version 2 restore needed 0 00000080:00200000:2.0:1761225263.971430:0:20347:0:(vvp_io.c:313:vvp_io_fini()) [0x200000402:0x30:0x0] ignore/verify layout 0/1, layout version 2 need write layout 0, restore needed 0 00010000:00010000:0.0:1761225264.496231:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1761225265.520077:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1761225266.545741:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1761225267.573443:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1761225268.594734:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225269.041055:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225269.041063:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225269.041067:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225269.041079:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 862 00000100:00080000:0.0:1761225269.041083:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1761225269.041090:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225269.041099:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225269.041114:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720587456/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225269.041135:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225269.041138:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225269.041187:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720587456:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225269.041198:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e861c82 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720587264:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225269.041291:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720587392:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225269.041322:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761225269.041327:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720587328:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225269.045750:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e861c82 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720587264:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225269.049072:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720587328:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225269.049096:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720587392:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225269.049193:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225269.049199:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225269.049204:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720587456:192.168.202.114@tcp:8:kworker/u8:1.0 00010000:00010000:1.0:1761225269.620500:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1761225270.640143:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000080:0.0:1761225270.857028:0:21049:0:(module.c:221:libcfs_ioctl()) libcfs ioctl cmd 3221775648 00000001:02000400:0.0:1761225270.857035:0:21049:0:(debug.c:702:libcfs_debug_mark_buffer()) DEBUG MARKER: == sanity-flr test 32: data should be mirrored to newly created mirror ========================================================== 09:14:29 (1761225269) 00000080:00200000:0.0:1761225270.960173:0:21053:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31),name=/ 00000002:00010000:0.0:1761225270.960302:0:21053:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:0.0:1761225270.960326:0:21053:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761225270.960351:0:21053:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00010000:0.0:1761225270.960364:0:21053:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) 00000080:00010000:0.0:1761225270.960368:0:21053:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) for lock 0xc207e904967bdbe5 00000080:00010000:0.0:1761225270.960376:0:21053:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdbe5 from it 000000001e1708df 00010000:00010000:0.0:1761225270.960379:0:21053:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761225270.960387:0:21053:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1761225270.960407:0:21053:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31), inode mode 41ed mask 1 00000080:00200000:0.0:1761225270.960421:0:21053:0:(namei.c:1107:ll_lookup_nd()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), flags=8 00000080:00200000:0.0:1761225270.960426:0:21053:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), intent=getattr 00000002:00010000:0.0:1761225270.960441:0:21053:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1761225270.960498:0:21053:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761225270.960505:0:21053:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761225270.960515:0:21053:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1761225270.960551:0:21053:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job rm:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21053:1846777720587520:192.168.202.114@tcp:101:rm.0 00000100:00100000:0.0:1761225270.961131:0:21053:0:(client.c:2500:ptlrpc_set_wait()) set 00000000675b64db going to sleep for 405 seconds 00000100:00100000:1.0:1761225270.967513:0:21053:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job rm:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21053:1846777720587520:192.168.202.114@tcp:101:rm.0 00010000:00010000:1.0:1761225270.967543:0:21053:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967572:0:21053:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967580:0:21053:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967598:0:21053:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967607:0:21053:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967614:0:21053:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000005b07afbc already unused, calling callback (00000000bafbdba0) 00010000:00010000:1.0:1761225270.967619:0:21053:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967640:0:21053:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967648:0:21053:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967657:0:21053:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1761225270.967664:0:21053:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967671:0:21053:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000005b07afbc/0xc207e904967bdc24 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1761225270.967693:0:21053:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@0000000052859013 x1846777720587520/t0(0) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 584/640 e 0 to 0 dl 1761225677 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'rm.0' 00010000:00010000:1.0:1761225270.967739:0:21053:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000000d364156 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:1.0:1761225270.967833:0:21053:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31),name=/ 00000002:00010000:1.0:1761225270.967885:0:21053:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:1.0:1761225270.967894:0:21053:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967906:0:21053:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00010000:1.0:1761225270.967915:0:21053:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) 00000080:00010000:1.0:1761225270.967919:0:21053:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) for lock 0xc207e904967bdbe5 00000080:00010000:1.0:1761225270.967925:0:21053:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdbe5 from it 00000000b1d67452 00010000:00010000:1.0:1761225270.967927:0:21053:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.967942:0:21053:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00200000:1.0:1761225270.967959:0:21053:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31), inode mode 41ed mask 1 00000080:00200000:1.0:1761225270.967975:0:21053:0:(namei.c:1107:ll_lookup_nd()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), flags=0 00000080:00200000:1.0:1761225270.967979:0:21053:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), intent=getattr 00000002:00010000:1.0:1761225270.967992:0:21053:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:1.0:1761225270.968044:0:21053:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.968050:0:21053:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225270.968061:0:21053:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1761225270.968085:0:21053:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job rm:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21053:1846777720587584:192.168.202.114@tcp:101:rm.0 00000100:00100000:1.0:1761225270.968176:0:21053:0:(client.c:2500:ptlrpc_set_wait()) set 00000000f13079ef going to sleep for 405 seconds 00000100:00100000:2.0:1761225270.979413:0:21053:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job rm:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21053:1846777720587584:192.168.202.114@tcp:101:rm.0 00010000:00010000:2.0:1761225270.979442:0:21053:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979459:0:21053:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979465:0:21053:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979472:0:21053:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979480:0:21053:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979487:0:21053:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000ad7b8b45 already unused, calling callback (00000000bafbdba0) 00010000:00010000:2.0:1761225270.979502:0:21053:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979509:0:21053:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979517:0:21053:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979524:0:21053:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1761225270.979530:0:21053:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979537:0:21053:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000ad7b8b45/0xc207e904967bdc2b lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1761225270.979564:0:21053:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@0000000017ee9697 x1846777720587584/t0(0) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 584/640 e 0 to 0 dl 1761225677 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'rm.0' 00010000:00010000:2.0:1761225270.979611:0:21053:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000000d364156 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:2.0:1761225270.979688:0:21053:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31),name=/ 00000002:00010000:2.0:1761225270.979703:0:21053:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1761225270.979709:0:21053:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979720:0:21053:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00010000:2.0:1761225270.979729:0:21053:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) 00000080:00010000:2.0:1761225270.979733:0:21053:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) for lock 0xc207e904967bdbe5 00000080:00010000:2.0:1761225270.979739:0:21053:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdbe5 from it 000000001e1708df 00010000:00010000:2.0:1761225270.979748:0:21053:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979754:0:21053:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761225270.979771:0:21053:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31), inode mode 41ed mask 1 00000080:00200000:2.0:1761225270.979793:0:21053:0:(namei.c:1107:ll_lookup_nd()) VFS Op:name=f32.sanity-flr-2, dir=[0x200000007:0x1:0x0](00000000d624db31), flags=8 00000080:00200000:2.0:1761225270.979798:0:21053:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr-2, dir=[0x200000007:0x1:0x0](00000000d624db31), intent=getattr 00000002:00010000:2.0:1761225270.979812:0:21053:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr-2,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:2.0:1761225270.979896:0:21053:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979903:0:21053:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225270.979913:0:21053:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1761225270.979939:0:21053:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job rm:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21053:1846777720587648:192.168.202.114@tcp:101:rm.0 00000100:00100000:2.0:1761225270.980030:0:21053:0:(client.c:2500:ptlrpc_set_wait()) set 0000000011985444 going to sleep for 405 seconds 00000100:00100000:3.0:1761225270.991372:0:21053:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job rm:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21053:1846777720587648:192.168.202.114@tcp:101:rm.0 00010000:00010000:3.0:1761225270.991400:0:21053:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991418:0:21053:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991425:0:21053:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991432:0:21053:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991441:0:21053:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991447:0:21053:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000009c7616c2 already unused, calling callback (00000000bafbdba0) 00010000:00010000:3.0:1761225270.991452:0:21053:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991469:0:21053:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991477:0:21053:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991485:0:21053:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1761225270.991491:0:21053:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991498:0:21053:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 000000009c7616c2/0xc207e904967bdc32 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1761225270.991514:0:21053:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@00000000b3214ada x1846777720587648/t0(0) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 592/640 e 0 to 0 dl 1761225677 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'rm.0' 00010000:00010000:3.0:1761225270.991562:0:21053:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000000d364156 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:3.0:1761225270.991626:0:21053:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31),name=/ 00000002:00010000:3.0:1761225270.991642:0:21053:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:3.0:1761225270.991649:0:21053:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991660:0:21053:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00010000:3.0:1761225270.991670:0:21053:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) 00000080:00010000:3.0:1761225270.991674:0:21053:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) for lock 0xc207e904967bdbe5 00000080:00010000:3.0:1761225270.991680:0:21053:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdbe5 from it 00000000b1d67452 00010000:00010000:3.0:1761225270.991682:0:21053:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991695:0:21053:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00200000:3.0:1761225270.991710:0:21053:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31), inode mode 41ed mask 1 00000080:00200000:3.0:1761225270.991732:0:21053:0:(namei.c:1107:ll_lookup_nd()) VFS Op:name=f32.sanity-flr-2, dir=[0x200000007:0x1:0x0](00000000d624db31), flags=0 00000080:00200000:3.0:1761225270.991737:0:21053:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr-2, dir=[0x200000007:0x1:0x0](00000000d624db31), intent=getattr 00000002:00010000:3.0:1761225270.991750:0:21053:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr-2,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:3.0:1761225270.991796:0:21053:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991802:0:21053:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.991812:0:21053:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1761225270.991837:0:21053:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job rm:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21053:1846777720587712:192.168.202.114@tcp:101:rm.0 00000100:00100000:3.0:1761225270.991951:0:21053:0:(client.c:2500:ptlrpc_set_wait()) set 0000000048a301a8 going to sleep for 405 seconds 00000100:00100000:3.0:1761225270.998572:0:21053:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job rm:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21053:1846777720587712:192.168.202.114@tcp:101:rm.0 00010000:00010000:3.0:1761225270.998613:0:21053:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.998635:0:21053:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.998643:0:21053:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.998663:0:21053:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.998670:0:21053:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.998676:0:21053:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000947af241 already unused, calling callback (00000000bafbdba0) 00010000:00010000:3.0:1761225270.998680:0:21053:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.998687:0:21053:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.998695:0:21053:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.998702:0:21053:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:3.0:1761225270.998709:0:21053:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1761225270.998715:0:21053:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000947af241/0xc207e904967bdc39 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21053 timeout: 0 lvb_type: 0 00000002:00100000:3.0:1761225270.998736:0:21053:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@00000000abb9f4e4 x1846777720587712/t0(0) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 592/640 e 0 to 0 dl 1761225677 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'rm.0' 00010000:00010000:3.0:1761225270.998779:0:21053:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000000d364156 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:0.0:1761225271.038260:0:21061:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31),name=/ 00000002:00010000:0.0:1761225271.038298:0:21061:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:0.0:1761225271.038318:0:21061:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761225271.038334:0:21061:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00010000:0.0:1761225271.038346:0:21061:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) 00000080:00010000:0.0:1761225271.038350:0:21061:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) for lock 0xc207e904967bdbe5 00000080:00010000:0.0:1761225271.038358:0:21061:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdbe5 from it 00000000433d0b83 00010000:00010000:0.0:1761225271.038361:0:21061:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761225271.038371:0:21061:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00200000:0.0:1761225271.038388:0:21061:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31), inode mode 41ed mask 1 00000080:00200000:0.0:1761225271.038400:0:21061:0:(namei.c:1107:ll_lookup_nd()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), flags=8 00000080:00200000:0.0:1761225271.038405:0:21061:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), intent=getattr 00000002:00010000:0.0:1761225271.038419:0:21061:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x0:0x0:0x0]) in obj [0x200000007:0x1:0x0], intent: getattr flags 0 00010000:00010000:0.0:1761225271.038472:0:21061:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761225271.038479:0:21061:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761225271.038489:0:21061:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1761225271.038529:0:21061:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720587776:192.168.202.114@tcp:101:lfs.0 00000100:00100000:0.0:1761225271.038654:0:21061:0:(client.c:2500:ptlrpc_set_wait()) set 00000000ac8a37e6 going to sleep for 405 seconds 00000100:00100000:2.0:1761225271.043321:0:21061:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720587776:192.168.202.114@tcp:101:lfs.0 00010000:00010000:2.0:1761225271.043350:0:21061:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043367:0:21061:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043374:0:21061:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043392:0:21061:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043401:0:21061:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043406:0:21061:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000036b49afa already unused, calling callback (00000000bafbdba0) 00010000:00010000:2.0:1761225271.043410:0:21061:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043416:0:21061:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043423:0:21061:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043430:0:21061:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:2.0:1761225271.043436:0:21061:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043443:0:21061:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 0000000036b49afa/0xc207e904967bdc40 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1761225271.043460:0:21061:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=8 disposition=7, status=0 req@00000000243e8b76 x1846777720587776/t0(0) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 584/640 e 0 to 0 dl 1761225678 ref 1 fl Complete:RQU/0/0 rc 301/301 job:'lfs.0' 00010000:00010000:2.0:1761225271.043497:0:21061:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000000d364156 type 13 mode 30 res 8589934599/1 (0 0) 00000080:00200000:2.0:1761225271.043648:0:21061:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31),name=/ 00000002:00010000:2.0:1761225271.043666:0:21061:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1761225271.043680:0:21061:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043690:0:21061:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00010000:2.0:1761225271.043700:0:21061:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) 00000080:00010000:2.0:1761225271.043703:0:21061:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) for lock 0xc207e904967bdbe5 00000080:00010000:2.0:1761225271.043710:0:21061:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdbe5 from it 00000000049a4271 00010000:00010000:2.0:1761225271.043712:0:21061:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043720:0:21061:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761225271.043736:0:21061:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31), inode mode 41ed mask 1 00000080:00200000:2.0:1761225271.043753:0:21061:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31),name=/ 00000002:00010000:2.0:1761225271.043759:0:21061:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1761225271.043764:0:21061:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043771:0:21061:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00010000:2.0:1761225271.043778:0:21061:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) 00000080:00010000:2.0:1761225271.043780:0:21061:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) for lock 0xc207e904967bdbe5 00000080:00010000:2.0:1761225271.043783:0:21061:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdbe5 from it 00000000000089c3 00010000:00010000:2.0:1761225271.043784:0:21061:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043800:0:21061:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761225271.043807:0:21061:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31), inode mode 41ed mask 3 00000080:00200000:2.0:1761225271.043812:0:21061:0:(namei.c:1176:ll_atomic_open()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), file 00000000acda0f9c, open_flags a141, mode 81b6 opened 0 00000080:00200000:2.0:1761225271.043822:0:21061:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), intent=open|creat 00000002:00010000:2.0:1761225271.043842:0:21061:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x200000402:0x31:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0120502 00010000:00010000:2.0:1761225271.043978:0:21061:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.043986:0:21061:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000020:00100000:2.0:1761225271.043997:0:21061:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff98ca898c0800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1761225271.044002:0:21061:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@00000000f5134a53 x1846777720587904/t0(0) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 528/67832 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' 00010000:00010000:2.0:1761225271.044009:0:21061:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1761225271.044031:0:21061:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720587904:192.168.202.114@tcp:101:lfs.0 00000100:00100000:2.0:1761225271.044178:0:21061:0:(client.c:2500:ptlrpc_set_wait()) set 0000000011985444 going to sleep for 405 seconds 00000100:00100000:2.0:1761225271.053317:0:21061:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720587904:192.168.202.114@tcp:101:lfs.0 00010000:00010000:2.0:1761225271.053346:0:21061:0:(ldlm_request.c:714:ldlm_cli_enqueue_fini()) ### server returned different mode CR ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053370:0:21061:0:(ldlm_request.c:729:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 4/0,1 mode: --/CR res: [0x200000402:0x31:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053387:0:21061:0:(ldlm_request.c:742:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 4/0,1 mode: --/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053397:0:21061:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 4/0,1 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053409:0:21061:0:(ldlm_request.c:796:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 4/0,1 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053421:0:21061:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 5/1,1 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053428:0:21061:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 5/1,1 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053435:0:21061:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1761225271.053443:0:21061:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=3 disposition=8000037, status=0 req@00000000f5134a53 x1846777720587904/t4294967752(4294967752) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 528/600 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'lfs.0' 00000002:00100000:2.0:1761225271.053455:0:21061:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@00000000f5134a53 x1846777720587904/t4294967752(4294967752) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 528/600 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'lfs.0' 00000002:00010000:2.0:1761225271.053464:0:21061:0:(mdc_locks.c:1179:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053475:0:21061:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000000d364156 type 13 mode 16 res 8589935618/49 (0 0) 00000080:00200000:2.0:1761225271.053494:0:21061:0:(namei.c:1405:ll_create_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), intent=open|creat 00000080:00200000:2.0:1761225271.053529:0:21061:0:(llite_lib.c:2833:ll_read_inode2()) VFS Op:inode=[0x200000402:0x31:0x0](00000000c41a2718) 00000080:00200000:2.0:1761225271.053535:0:21061:0:(llite_lib.c:2755:ll_update_inode()) inode=[0x200000402:0x31:0x0], updating i_size 0 00000080:00200000:2.0:1761225271.053584:0:21061:0:(file.c:5870:ll_layout_conf()) [0x200000402:0x31:0x0]: layout version change: 4294967294 -> 4294967295 00000080:00010000:2.0:1761225271.053590:0:21061:0:(namei.c:1372:ll_create_node()) setting l_ast_data to inode [0x200000402:0x31:0x0](00000000c41a2718) 00000080:00010000:2.0:1761225271.053593:0:21061:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000402:0x31:0x0](00000000c41a2718) for lock 0xc207e904967bdc47 00000080:00200000:2.0:1761225271.053606:0:21061:0:(file.c:837:ll_file_open()) VFS Op:inode=[0x200000402:0x31:0x0](00000000c41a2718), flags 120501 00000002:00100000:2.0:1761225271.053635:0:21061:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@00000000f5134a53 x1846777720587904/t4294967752(4294967752) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 528/600 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'lfs.0' 00000080:00010000:2.0:1761225271.053731:0:21061:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdc47 from it 000000006f9f9740 00010000:00010000:2.0:1761225271.053735:0:21061:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053743:0:21061:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761225271.053760:0:21061:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000402:0x31:0x0](00000000c41a2718), inode mode 81a4 mask 40 00000080:00200000:2.0:1761225271.053799:0:21061:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000402:0x31:0x0](00000000c41a2718), inode mode 81a4 mask 2 00000080:00200000:2.0:1761225271.053809:0:21061:0:(xattr.c:358:ll_xattr_set()) VFS Op:inode=[0x200000402:0x31:0x0](00000000c41a2718), xattr lov 00000002:00010000:2.0:1761225271.053826:0:21061:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000402:0x31:0x0]) in obj [0x200000007:0x1:0x0], intent: open flags 040000002 00010000:00010000:2.0:1761225271.053931:0:21061:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.053939:0:21061:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000020:00100000:2.0:1761225271.053949:0:21061:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff98ca898c0800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1761225271.053955:0:21061:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@00000000a37cdda5 x1846777720588032/t0(0) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 2216/133368 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' 00010000:00010000:2.0:1761225271.053962:0:21061:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1761225271.053988:0:21061:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720588032:192.168.202.114@tcp:101:lfs.0 00000100:00100000:2.0:1761225271.054148:0:21061:0:(client.c:2500:ptlrpc_set_wait()) set 0000000011985444 going to sleep for 405 seconds 00000100:00100000:0.0:1761225271.060532:0:2277:0:(events.c:361:request_in_callback()) peer: 12345-192.168.202.114@tcp (source: 12345-192.168.202.114@tcp) 00000100:00100000:0.0:1761225271.060596:0:5722:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1846777737344064 00000100:00100000:0.0:1761225271.060761:0:5722:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.202.114@tcp, seq: 332 00000100:00100000:0.0:1761225271.060768:0:5722:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e861c82 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:774c7a2d-e1c6-4d57-85f8-be532a26f7be+5:9313:x1846777737344064:12345-192.168.202.114@tcp:104: 00010000:00010000:0.0:1761225271.060785:0:5722:0:(ldlm_lockd.c:2468:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1761225271.060907:0:5722:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:0.0:1761225271.060913:0:5722:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e861c82 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_000:774c7a2d-e1c6-4d57-85f8-be532a26f7be+5:9313:x1846777737344064:12345-192.168.202.114@tcp:104: Request processed in 145us (390us total) trans 0 rc 0/0 00010000:00010000:2.0:1761225271.060913:0:5733:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1761225271.060918:0:5733:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x420000000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1761225271.060920:0:5722:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.202.114@tcp, seq: 332 00010000:00010000:2.0:1761225271.060926:0:5733:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000d0eeefb8 already unused, calling callback (00000000bafbdba0) 00010000:00010000:2.0:1761225271.060930:0:5733:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.060938:0:5733:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x420400000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.060944:0:5733:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 3/0,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x428400000000 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761225271.060977:0:5733:0:(vvp_object.c:137:vvp_conf_set()) [0x200000402:0x31:0x0]: losing layout lock 00010000:00010000:2.0:1761225271.061201:0:5733:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 2/0,0 mode: --/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.061209:0:5733:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:2.0:1761225271.061230:0:5733:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 1/0,0 mode: --/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.061236:0:5733:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc47 lrc: 0/0,0 mode: --/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x9 rrc: 2 type: IBT gid 0 flags: 0x4c29400000020 nid: local remote: 0xaa1cd6a2dbef0c78 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.061261:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1761225271.061281:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1761225271.061282:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225271.061364:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028d086a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720588096:192.168.202.114@tcp:103:ldlm_bl_02.0 00000100:00100000:1.0:1761225271.065658:0:21061:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720588032:192.168.202.114@tcp:101:lfs.0 00010000:00010000:1.0:1761225271.065683:0:21061:0:(ldlm_request.c:641:ldlm_cli_enqueue_fini()) ### client-side enqueue END (ABORTED) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225271.065700:0:21061:0:(ldlm_request.c:562:failed_lock_cleanup()) ### setting FL_LOCAL_ONLY | LDLM_FL_FAILED | LDLM_FL_ATOMIC_CB | LDLM_FL_CBPENDING ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225271.065708:0:21061:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225271.065733:0:21061:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x10206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225271.065743:0:21061:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225271.065750:0:21061:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000d7e0fedd already unused, calling callback (00000000bafbdba0) 00010000:00010000:1.0:1761225271.065754:0:21061:0:(ldlm_request.c:1546:ldlm_cli_convert()) ### client lock convert START ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225271.065761:0:21061:0:(ldlm_request.c:1556:ldlm_cli_convert()) ### client lock convert END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 4/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x206400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225271.065769:0:21061:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 5/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x20e400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1761225271.065823:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028d086a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720588096:192.168.202.114@tcp:103:ldlm_bl_02.0 00010000:00010000:1.0:1761225271.066086:0:21061:0:(ldlm_request.c:1248:ldlm_cli_cancel_local()) not sending request (at caller's instruction) 00010000:00010000:1.0:1761225271.066092:0:21061:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 3/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1761225271.066099:0:21061:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d7e0fedd/0xc207e904967bdc4e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x1 rrc: 3 type: IBT gid 0 flags: 0x4a0f400000000 nid: local remote: 0x0 expref: -99 pid: 21061 timeout: 0 lvb_type: 0 00000002:00100000:1.0:1761225271.066116:0:21061:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=1 disposition=800002b, status=0 req@00000000a37cdda5 x1846777720588032/t4294967753(4294967753) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 2216/736 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 301/301 job:'lfs.0' 00000002:00100000:1.0:1761225271.066129:0:21061:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@00000000a37cdda5 x1846777720588032/t4294967753(4294967753) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 2216/736 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 301/301 job:'lfs.0' 00000080:00200000:1.0:1761225271.066191:0:21061:0:(vvp_io.c:1820:vvp_io_init()) [0x200000402:0x31:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00000080:00200000:1.0:1761225271.066207:0:21061:0:(vvp_io.c:313:vvp_io_fini()) [0x200000402:0x31:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0 00000080:00200000:1.0:1761225271.066253:0:21061:0:(llite_lib.c:2755:ll_update_inode()) inode=[0x200000402:0x31:0x0], updating i_size 0 00000002:00100000:1.0:1761225271.066269:0:21061:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@00000000a37cdda5 x1846777720588032/t4294967753(4294967753) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 656/736 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 301/301 job:'lfs.0' 00000002:00100000:1.0:1761225271.066291:0:21061:0:(mdc_request.c:944:mdc_close()) @@@ matched open req@00000000a37cdda5 x1846777720588032/t4294967753(4294967753) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 656/736 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 301/301 job:'lfs.0' 00000020:00100000:1.0:1761225271.066313:0:21061:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff98ca898c0800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:1.0:1761225271.066319:0:21061:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@0000000057d87282 x1846777720588224/t0(0) o35->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:23/10 lens 392/2192 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761225271.066339:0:21061:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720588224:192.168.202.114@tcp:35:lfs.0 00000100:00100000:1.0:1761225271.066408:0:21061:0:(client.c:2500:ptlrpc_set_wait()) set 00000000f13079ef going to sleep for 6 seconds 00000100:00100000:0.0:1761225271.084664:0:21061:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720588224:192.168.202.114@tcp:35:lfs.0 00000002:00100000:0.0:1761225271.084701:0:21061:0:(mdc_request.c:854:mdc_free_open()) @@@ free open request, rq_replay=0 req@00000000a37cdda5 x1846777720588032/t4294967753(4294967753) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 656/736 e 0 to 0 dl 1761225678 ref 2 fl Complete:RQU/4/0 rc 301/301 job:'lfs.0' 00000080:00200000:0.0:1761225271.084752:0:21061:0:(file.c:410:ll_file_release()) VFS Op:inode=[0x200000402:0x31:0x0](00000000c41a2718) 00000002:00100000:0.0:1761225271.084790:0:21061:0:(mdc_request.c:944:mdc_close()) @@@ matched open req@00000000f5134a53 x1846777720587904/t4294967752(4294967752) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 528/600 e 0 to 0 dl 1761225678 ref 1 fl Complete:RPQU/4/ffffffff rc 0/-1 job:'lfs.0' 00000020:00100000:0.0:1761225271.084815:0:21061:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff98ca898c0800: modify RPC slot 1 is allocated opc 35, max 7 00000100:00100000:0.0:1761225271.084821:0:21061:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@0000000035650445 x1846777720588352/t0(0) o35->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:23/10 lens 392/2192 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225271.084843:0:21061:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720588352:192.168.202.114@tcp:35:lfs.0 00000100:00100000:0.0:1761225271.085747:0:21061:0:(client.c:2500:ptlrpc_set_wait()) set 00000000fc9691af going to sleep for 6 seconds 00000100:00100000:0.0:1761225271.092574:0:21061:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job lfs:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21061:1846777720588352:192.168.202.114@tcp:35:lfs.0 00000002:00100000:0.0:1761225271.092609:0:21061:0:(mdc_request.c:854:mdc_free_open()) @@@ free open request, rq_replay=0 req@00000000f5134a53 x1846777720587904/t4294967752(4294967752) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 528/600 e 0 to 0 dl 1761225678 ref 1 fl Complete:RQU/4/ffffffff rc 0/-1 job:'lfs.0' 00000080:00200000:2.0:1761225271.107355:0:21065:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31),name=/ 00000002:00010000:2.0:1761225271.107385:0:21065:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1761225271.107402:0:21065:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.107415:0:21065:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00010000:2.0:1761225271.107426:0:21065:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) 00000080:00010000:2.0:1761225271.107430:0:21065:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) for lock 0xc207e904967bdbe5 00000080:00010000:2.0:1761225271.107436:0:21065:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdbe5 from it 000000006a7342ea 00010000:00010000:2.0:1761225271.107439:0:21065:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.107446:0:21065:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761225271.107460:0:21065:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31), inode mode 41ed mask 1 00000080:00200000:2.0:1761225271.107475:0:21065:0:(file.c:5227:ll_inode_revalidate()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31),name=/ 00000002:00010000:2.0:1761225271.107480:0:21065:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: ,[0x200000007:0x1:0x0]) in obj [0x200000007:0x1:0x0], intent: lookup flags 0 00010000:00010000:2.0:1761225271.107484:0:21065:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.107490:0:21065:0:(ldlm_lock.c:1494:ldlm_lock_match_with_skip()) ### matched (0 0) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00010000:2.0:1761225271.107497:0:21065:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) 00000080:00010000:2.0:1761225271.107498:0:21065:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000007:0x1:0x0](00000000d624db31) for lock 0xc207e904967bdbe5 00000080:00010000:2.0:1761225271.107509:0:21065:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdbe5 from it 000000005402813e 00010000:00010000:2.0:1761225271.107511:0:21065:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.107518:0:21065:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000cc9d407d/0xc207e904967bdbe5 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0b36 expref: -99 pid: 19254 timeout: 0 lvb_type: 0 00000080:00200000:2.0:1761225271.107525:0:21065:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000007:0x1:0x0](00000000d624db31), inode mode 41ed mask 3 00000080:00200000:2.0:1761225271.107530:0:21065:0:(namei.c:1176:ll_atomic_open()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), file 000000001b51838b, open_flags 8241, mode 81b6 opened 0 00000080:00200000:2.0:1761225271.107539:0:21065:0:(namei.c:833:ll_lookup_it()) VFS Op:name=f32.sanity-flr, dir=[0x200000007:0x1:0x0](00000000d624db31), intent=open|creat 00000002:00010000:2.0:1761225271.107568:0:21065:0:(mdc_locks.c:1328:mdc_intent_lock()) (name: f32.sanity-flr,[0x200000402:0x32:0x0]) in obj [0x200000007:0x1:0x0], intent: open|creat flags 0101102 00010000:00010000:2.0:1761225271.107625:0:21065:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.107632:0:21065:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000 ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00000020:00100000:2.0:1761225271.107643:0:21065:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff98ca898c0800: modify RPC slot 1 is allocated opc 101, max 7 00000100:00100000:2.0:1761225271.107654:0:21065:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@0000000076b78222 x1846777720588480/t0(0) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 2224/67832 e 0 to 0 dl 0 ref 1 fl New:PQU/0/ffffffff rc 0/-1 job:'' 00010000:00010000:2.0:1761225271.107662:0:21065:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1761225271.107684:0:21065:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000076b78222 pname:cluuid:pid:xid:nid:opc:job dd:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21065:1846777720588480:192.168.202.114@tcp:101:dd.0 00000100:00100000:2.0:1761225271.107845:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 000000000c7b8d48 going to sleep for 405 seconds 00000100:00100000:2.0:1761225271.113364:0:21065:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000076b78222 pname:cluuid:pid:xid:nid:opc:job dd:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21065:1846777720588480:192.168.202.114@tcp:101:dd.0 00010000:00010000:2.0:1761225271.113390:0:21065:0:(ldlm_request.c:714:ldlm_cli_enqueue_fini()) ### server returned different mode CR ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 4/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.113430:0:21065:0:(ldlm_request.c:729:ldlm_cli_enqueue_fini()) ### client-side enqueue, new resource ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 4/0,1 mode: --/CR res: [0x200000402:0x31:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.113437:0:21065:0:(ldlm_request.c:742:ldlm_cli_enqueue_fini()) ### client-side enqueue, new policy data ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 4/0,1 mode: --/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.113446:0:21065:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 4/0,1 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.113456:0:21065:0:(ldlm_request.c:796:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 4/0,1 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.113468:0:21065:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 5/1,1 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.113474:0:21065:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 5/1,1 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1761225271.113480:0:21065:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 4/1,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00000002:00100000:2.0:1761225271.113489:0:21065:0:(mdc_locks.c:743:mdc_finish_enqueue()) @@@ op=3 disposition=2b, status=0 req@0000000076b78222 x1846777720588480/t4294967756(4294967756) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 2224/736 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'dd.0' 00000002:00100000:2.0:1761225271.113502:0:21065:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@0000000076b78222 x1846777720588480/t4294967756(4294967756) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 2224/736 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'dd.0' 00000002:00010000:2.0:1761225271.113513:0:21065:0:(mdc_locks.c:847:mdc_finish_enqueue()) ### layout lock returned by: open|creat, lvb_len: 136 ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 0 00000002:00010000:2.0:1761225271.113538:0:21065:0:(mdc_locks.c:1179:mdc_finish_intent_lock()) ### matching against this ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1761225271.113548:0:21065:0:(ldlm_lock.c:1511:ldlm_lock_match_with_skip()) ### not matched ns 000000000d364156 type 13 mode 16 res 8589935618/49 (0 0) 00000080:00200000:2.0:1761225271.113754:0:21065:0:(namei.c:157:ll_iget()) got inode: [0x200000402:0x31:0x0](00000000c41a2718): rc = 0 00000080:00200000:2.0:1761225271.113768:0:21065:0:(file.c:5870:ll_layout_conf()) [0x200000402:0x31:0x0]: layout version change: 4294967294 -> 1 00000080:00010000:2.0:1761225271.113774:0:21065:0:(llite_internal.h:1711:ll_set_lock_data()) setting l_data to inode [0x200000402:0x31:0x0](00000000c41a2718) for lock 0xc207e904967bdc55 00000080:00010000:2.0:1761225271.113838:0:21065:0:(dcache.c:293:ll_lookup_finish_locks()) setting l_data to inode [0x200000402:0x31:0x0](00000000c41a2718) 00000080:00200000:2.0:1761225271.114153:0:21065:0:(file.c:837:ll_file_open()) VFS Op:inode=[0x200000402:0x31:0x0](00000000c41a2718), flags 101101 00000002:00100000:2.0:1761225271.114180:0:21065:0:(mdc_request.c:833:mdc_set_open_replay_data()) @@@ Set up open replay data req@0000000076b78222 x1846777720588480/t4294967756(4294967756) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 664/736 e 0 to 0 dl 1761225678 ref 2 fl Complete:RPQU/4/0 rc 0/0 job:'dd.0' 00000080:00010000:2.0:1761225271.114263:0:21065:0:(dcache.c:193:ll_intent_drop_lock()) releasing lock with cookie 0xc207e904967bdc55 from it 000000006f9f9740 00010000:00010000:2.0:1761225271.114267:0:21065:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 3/1,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x10020000000000 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 3 00010000:00010000:2.0:1761225271.114275:0:21065:0:(ldlm_lock.c:902:ldlm_lock_decref_internal()) ### add lock into lru list ns: lustre-MDT0000-mdc-ffff98ca898c0800 lock: 00000000d0eeefb8/0xc207e904967bdc55 lrc: 2/0,0 mode: CR/CR res: [0x200000402:0x31:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x20000000000 nid: local remote: 0xaa1cd6a2dbef0cb0 expref: -99 pid: 21065 timeout: 0 lvb_type: 3 00000080:00200000:2.0:1761225271.114292:0:21065:0:(file.c:5661:ll_inode_permission()) VFS Op:inode=[0x200000402:0x31:0x0](00000000c41a2718), inode mode 81a4 mask 42 00000080:00200000:2.0:1761225271.114306:0:21065:0:(llite_lib.c:2191:ll_setattr_raw()) lustre: setattr inode [0x200000402:0x31:0x0](00000000c41a2718) from 0 to 0, valid a068, hsm_import 0 00000020:00100000:2.0:1761225271.114348:0:21065:0:(genops.c:2339:obd_get_mod_rpc_slot()) lustre-MDT0000-mdc-ffff98ca898c0800: modify RPC slot 1 is allocated opc 36, max 7 00000100:00100000:2.0:1761225271.114354:0:21065:0:(client.c:742:ptlrpc_reassign_next_xid()) @@@ reassign xid req@0000000046c2349c x1846777720588608/t0(0) o36->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 488/512 e 0 to 0 dl 0 ref 1 fl New:QU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761225271.114373:0:21065:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job dd:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21065:1846777720588608:192.168.202.114@tcp:36:dd.0 00000100:00100000:2.0:1761225271.114482:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 405 seconds 00000100:00100000:2.0:1761225271.117130:0:21065:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job dd:774c7a2d-e1c6-4d57-85f8-be532a26f7be:21065:1846777720588608:192.168.202.114@tcp:36:dd.0 00000080:00200000:2.0:1761225271.117192:0:21065:0:(file.c:1543:ll_io_set_mirror()) f32.sanity-flr: desiginated mirror: 0 00000080:00200000:2.0:1761225271.117198:0:21065:0:(vvp_io.c:1820:vvp_io_init()) [0x200000402:0x31:0x0] ignore/verify layout 0/1, layout version 2 restore needed 0 00020000:00200000:2.0:1761225271.117214:0:21065:0:(lov_io.c:825:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1761225271.117229:0:21065:0:(vvp_io.c:222:vvp_io_one_lock_index()) lock: 1 [0, 18446744073709551615] 00000008:00010000:2.0:1761225271.117242:0:21065:0:(lustre_osc.h:181:osc_enq2ldlm_flags()) flags: 4 00000008:00010000:2.0:1761225271.117247:0:21065:0:(osc_lock.c:1241:osc_lock_init()) ### lock 00000000465258e6, osc lock 000000000a9157d0, flags 0x80000000 00010000:00010000:2.0:1761225271.117282:0:21065:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000147fcfc2/0xc207e904967bdc5c lrc: 3/0,1 mode: --/PW res: [0x3d:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 21065 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1761225271.117293:0:21065:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x80000000 ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000147fcfc2/0xc207e904967bdc5c lrc: 3/0,1 mode: --/PW res: [0x3d:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21065 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1761225271.117321:0:21065:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-OST0000-osc-ffff98ca898c0800 lock: 00000000147fcfc2/0xc207e904967bdc5c lrc: 3/0,1 mode: --/PW res: [0x3d:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 21065 timeout: 0 lvb_type: 1 00000100:00080000:2.0:1761225271.117339:0:21065:0:(client.c:1659:ptlrpc_send_new_req()) @@@ req waiting for recovery: (FULL != DISCONN) req@0000000028d086a4 x1846777720588672/t0(0) o101->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 328/400 e 0 to 0 dl 0 ref 2 fl Rpc:WQU/0/ffffffff rc 0/-1 job:'dd.0' 00000100:00100000:2.0:1761225271.117349:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225271.664178:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225272.176014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225272.688206:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225273.200359:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225273.712046:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225274.161256:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225274.161266:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225274.161270:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225274.161283:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 867 00000100:00080000:0.0:1761225274.161289:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761225274.161298:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225274.161311:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225274.161331:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000243e8b76 x1846777720588928/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225274.161346:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225274.161349:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225274.161378:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720588800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225274.161483:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761225274.161489:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720588736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225274.161516:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720588864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225274.162503:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720588928:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225274.170838:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720588736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225274.179468:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720588864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225274.179517:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720588800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225274.185082:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225274.185098:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225274.185111:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720588928:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225274.224013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225274.736787:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225275.248032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225275.763828:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225276.272312:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225276.784007:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225277.296470:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225277.808067:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225278.319989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225278.832228:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225279.280898:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225279.280908:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225279.280913:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225279.280924:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 872 00000100:00080000:3.0:1761225279.280928:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761225279.280933:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225279.280943:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225279.280958:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720589184/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225279.280973:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225279.280976:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225279.281001:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225279.281114:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761225279.281122:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720588992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225279.281146:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225279.281703:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720589184:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225279.285740:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720588992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225279.285760:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225279.285771:0:2288:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0000-mdc-ffff98ca898c0800: committing for last_committed 4294967757 gen 1 00000100:00100000:3.0:1761225279.285933:0:2288:0:(client.c:2899:ptlrpc_free_committed()) @@@ keeping (FL_REPLAY) req@0000000076b78222 x1846777720588480/t4294967756(4294967756) o101->lustre-MDT0000-mdc-ffff98ca898c0800@192.168.202.114@tcp:12/10 lens 664/736 e 0 to 0 dl 1761225678 ref 1 fl Complete:RPQU/4/ffffffff rc 0/-1 job:'dd.0' 00000100:00100000:3.0:1761225279.285946:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225279.290751:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225279.290765:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225279.291039:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720589184:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225279.344558:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225279.856491:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225280.368109:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225280.880374:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225281.391968:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225281.904097:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225282.416103:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225282.930216:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225283.440188:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225283.952653:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225284.400116:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225284.400123:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225284.400127:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225284.400138:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 877 00000100:00080000:2.0:1761225284.400141:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761225284.400147:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225284.400158:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225284.400173:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec5c4aea x1846777720589440/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225284.400199:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225284.400203:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225284.400235:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720589312:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225284.400405:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720589248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225284.400587:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720589440:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225284.401974:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589376:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225284.410677:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720589248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225284.411263:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589376:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225284.411693:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720589312:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225284.411810:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225284.411816:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225284.411820:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720589440:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225284.463985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225284.977296:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225285.487966:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225285.999998:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225286.515083:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225287.024111:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225287.536038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225288.048841:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225288.560306:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225289.072012:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225289.520132:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225289.520140:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225289.520144:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225289.520154:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 882 00000100:00080000:3.0:1761225289.520158:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761225289.520164:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225289.520172:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225289.520185:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003d27cd56 x1846777720589696/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225289.520209:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225289.520212:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225289.520234:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225289.520329:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761225289.520334:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720589504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225289.520353:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225289.520470:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720589696:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225289.528230:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720589504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225289.528281:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225289.528292:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720589568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225289.529161:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225289.529177:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225289.529185:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720589696:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225289.584093:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225290.096710:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225290.607987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225291.120603:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225291.637673:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225292.144074:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225292.655986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225293.168707:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225293.680080:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225294.192343:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761225294.640242:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720589760:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225294.643369:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720589824:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225294.644937:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720589888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225294.646575:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225294.646586:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225294.646606:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225294.646617:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 887 00000100:00080000:0.0:1761225294.646621:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225294.647499:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225294.647514:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225294.647557:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720589952/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761225294.647637:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720589952:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761225294.654523:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225294.654534:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225294.658648:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720589760:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225294.661508:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720589824:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225294.661577:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720589888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225294.662163:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225294.662168:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225294.662172:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720589952:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225294.704035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225295.216075:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225295.729284:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225296.240599:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225296.753457:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225297.264181:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225297.777851:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225298.288093:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225298.799979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225299.312844:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225299.761741:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225299.761750:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225299.761754:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225299.761766:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 892 00000100:00080000:3.0:1761225299.761770:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225299.761776:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225299.761787:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225299.761803:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008536b8df x1846777720590208/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225299.761832:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225299.761835:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225299.763011:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720590208:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225299.763550:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720590016:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225299.763593:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720590144:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225299.763622:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225299.763628:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720590080:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225299.770249:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225299.770260:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225299.770269:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720590208:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225299.770489:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720590016:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225299.770530:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720590144:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225299.770563:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720590080:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225299.823996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225300.336742:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225300.851402:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225301.360242:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000080:00200000:2.0:1761225301.808018:0:36:0:(vvp_io.c:1820:vvp_io_init()) [0x200000402:0x31:0x0] ignore/verify layout 1/0, layout version 0 restore needed 0 00020000:00200000:2.0:1761225301.808037:0:36:0:(lov_io.c:825:lov_io_iter_init()) component[0] flags 0x10 00000080:00200000:2.0:1761225301.808063:0:36:0:(vvp_io.c:313:vvp_io_fini()) [0x200000402:0x31:0x0] ignore/verify layout 1/0, layout version 0 need write layout 0, restore needed 0 00000100:00100000:2.0:1761225301.873747:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225302.386428:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225302.896021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225303.409008:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225303.920095:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225304.432953:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225304.880149:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225304.880158:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225304.880162:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225304.880174:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 898 00000100:00080000:0.0:1761225304.880177:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761225304.880177:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720590272:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225304.880200:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225304.880211:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225304.880227:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720590464/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225304.880257:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225304.880260:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225304.880262:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720590400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225304.880281:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720590336:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225304.880384:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720590464:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225304.885459:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720590272:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225304.885501:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720590400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225304.885575:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720590336:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225304.885591:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225304.885596:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225304.885600:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720590464:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225304.943999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225305.456320:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225305.967974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225306.480058:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225306.992308:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225307.504004:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225308.015988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225308.530288:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225309.040362:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225309.552681:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225310.000112:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225310.000120:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225310.000125:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225310.000148:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 903 00000100:00080000:2.0:1761225310.000152:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225310.000159:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225310.000169:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225310.000188:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006be78503 x1846777720590720/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225310.000214:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225310.000218:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225310.000249:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720590720:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225310.000322:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720590528:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225310.000361:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720590656:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225310.000390:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225310.000398:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720590592:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225310.006832:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720590592:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225310.006922:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720590656:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225310.008189:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720590528:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225310.009292:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225310.009301:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225310.009308:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720590720:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225310.063984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225310.576132:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225311.088122:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225311.600329:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225312.113949:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225312.625290:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225313.135958:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225313.648007:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225314.159991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225314.672557:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225315.120083:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225315.120094:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00100000:3.0:1761225315.120114:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720590784:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225315.120115:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225315.120125:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 908 00000100:00080000:2.0:1761225315.120128:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225315.120134:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225315.120144:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225315.120162:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008451bff4 x1846777720590976/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761225315.120223:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720590912:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225315.120237:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720590976:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225315.120257:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225315.120266:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720590848:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225315.121889:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225315.121899:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225315.123281:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720590784:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225315.127805:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720590912:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225315.129433:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225315.129443:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225315.129453:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720590976:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225315.129624:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720590848:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225315.183989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225315.696094:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225316.207981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225316.720074:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225317.234027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225317.747562:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225318.256294:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225318.768069:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225319.279971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225319.792055:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761225320.242174:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720591040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225320.243617:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761225320.243632:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720591104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225320.243981:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225320.243987:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225320.244008:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225320.244014:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 913 00000100:00080000:0.0:1761225320.244017:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225320.244023:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225320.244036:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225320.244055:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000de75897f x1846777720591232/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761225320.244121:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720591232:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225320.244785:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720591168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225320.244954:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225320.244958:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225320.251549:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720591168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225320.251568:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720591104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225320.251578:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720591040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225320.251844:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225320.251850:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225320.251867:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720591232:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225320.303992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225320.816105:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225321.331681:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225321.840066:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225322.351995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225322.864112:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225323.376032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225323.888104:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225324.400047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225324.915684:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225325.360143:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225325.360152:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225325.360156:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225325.360167:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 918 00000100:00080000:3.0:1761225325.360171:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225325.360177:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225325.360187:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225325.360201:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002b7677b0 x1846777720591488/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225325.360230:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225325.360233:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225325.360261:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720591360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225325.360276:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720591488:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225325.360469:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720591296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225325.362608:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720591424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225325.371124:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720591360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225325.372187:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720591296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225325.372206:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720591424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225325.395741:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225325.395753:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225325.395763:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720591488:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225325.427908:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225325.936374:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225326.447989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225326.960422:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225327.472029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225327.985036:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225328.500687:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225329.008023:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225329.520302:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225330.033540:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225330.482915:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225330.482924:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225330.482928:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225330.482940:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 923 00000100:00080000:3.0:1761225330.482944:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225330.482952:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225330.482966:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225330.483003:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720591744/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225330.483028:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225330.483031:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225330.483341:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720591616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225330.483765:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720591744:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225330.486152:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720591552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225330.488119:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720591680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225330.492826:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720591552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225330.493544:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720591616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225330.493704:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225330.493710:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225330.493714:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720591744:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225330.494487:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720591680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225330.544140:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225331.060127:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225331.567998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225332.080429:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225332.593781:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225333.104056:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225333.616428:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225334.128219:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225334.640080:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225335.154119:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225335.600279:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720591808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225335.601073:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225335.601079:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225335.601084:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761225335.601091:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720591936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225335.601093:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 928 00000100:00080000:2.0:1761225335.601098:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225335.601107:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225335.601122:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225335.601142:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006bca6691 x1846777720592000/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761225335.601155:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225335.601163:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720591872:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225335.601203:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720592000:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761225335.601822:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225335.601828:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225335.605785:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720591808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225335.606114:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225335.606120:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225335.606126:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720592000:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225335.606217:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720591872:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225335.606230:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720591936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225335.664070:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225336.176255:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225336.688171:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225337.200168:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225337.712020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225338.224031:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225338.736026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225339.248057:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225339.768851:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225340.278919:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225340.721442:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225340.721452:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225340.721457:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225340.721468:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 933 00000100:00080000:0.0:1761225340.721472:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761225340.721475:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720592064:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225340.721481:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225340.721493:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225340.721511:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720592256/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225340.721543:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225340.721547:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225340.721569:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720592128:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225340.721586:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720592192:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225340.722032:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720592256:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225340.725117:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720592128:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225340.725166:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720592064:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225340.725197:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720592192:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:02020000:2.0:1761225340.732716:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:2.0:1761225340.741165:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225340.741177:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225340.741189:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720592256:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225340.783994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225341.296580:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225341.808117:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225342.320075:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225342.845351:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225343.344704:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225343.855991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225344.373550:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225344.880304:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225345.392848:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225345.846308:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225345.846333:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225345.846338:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225345.846352:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 938 00000100:00080000:0.0:1761225345.846356:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225345.846363:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225345.846375:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225345.846394:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720592512/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225345.846427:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225345.846448:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225345.846623:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720592512:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225345.846813:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720592320:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225345.846846:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720592384:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225345.846908:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720592448:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225345.850391:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720592384:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225345.850467:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720592448:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225345.850922:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225345.850928:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225345.850934:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720592512:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225345.851016:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720592320:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225345.904231:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225346.416613:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225346.927983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225347.440111:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225347.953645:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225348.464224:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225348.977144:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225349.488735:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225350.000470:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225350.512101:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225350.960111:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225350.960120:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225350.960123:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225350.960136:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 944 00000100:00080000:0.0:1761225350.960140:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225350.960146:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225350.960156:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225350.960170:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720592768/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225350.960196:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225350.960199:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225350.960314:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720592576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225350.960392:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720592704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225350.960416:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761225350.960421:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720592640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225350.962933:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720592768:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761225350.968484:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225350.968496:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225350.968517:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720592768:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225350.968621:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720592640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225350.968632:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720592576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225350.968641:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720592704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225351.024076:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225351.542878:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225352.047979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225352.562500:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225353.071995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225353.584067:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225354.097583:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225354.608076:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225355.120035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225355.632432:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225356.080107:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225356.080117:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225356.080123:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225356.080137:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 949 00000100:00080000:0.0:1761225356.080142:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761225356.080142:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720592832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225356.080153:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225356.080184:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225356.080202:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000024af3661 x1846777720593024/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225356.080232:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225356.080236:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225356.080246:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720592960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225356.080282:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761225356.080299:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720592896:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225356.080779:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720593024:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761225356.094079:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225356.094092:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225356.094103:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720593024:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225356.095157:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720592896:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225356.095173:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720592832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225356.095182:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720592960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225356.144000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225356.656740:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225357.168070:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225357.680467:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225358.192177:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225358.704083:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225359.215987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225359.728049:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225360.239996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225360.752186:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225361.200104:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720593088:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225361.201343:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225361.201350:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225361.201354:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225361.201361:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 954 00000100:00100000:0.0:1761225361.201366:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720593152:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225361.201373:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225361.201383:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225361.201396:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225361.201419:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720593280/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225361.201423:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225361.201433:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720593216:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225361.201443:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225361.201463:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225361.202186:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720593280:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225361.208066:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720593216:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225361.208083:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720593152:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225361.209346:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225361.209353:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225361.209360:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720593280:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225361.209469:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720593088:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225361.263976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225361.776882:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225362.287992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225362.801631:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225363.314445:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225363.824058:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225364.344384:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225364.848287:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225365.361453:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225365.872747:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225366.320147:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225366.320155:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225366.320159:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225366.320170:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 959 00000100:00080000:2.0:1761225366.320173:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225366.320179:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225366.320195:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225366.320211:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c987228a x1846777720593536/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225366.320242:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225366.320262:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225366.320351:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720593536:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225366.321328:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720593344:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225366.321396:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720593472:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225366.321426:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225366.321432:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720593408:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225366.326491:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720593408:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225366.326525:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720593472:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225366.327182:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720593344:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225366.327747:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225366.327753:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225366.327758:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720593536:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225366.384003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225366.899835:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225367.407980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225367.920120:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225368.432039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225368.946365:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225369.456005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225369.968043:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225370.480820:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225370.992451:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225371.441130:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225371.441139:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225371.441143:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761225371.441160:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720593600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225371.441168:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 964 00000100:00080000:0.0:1761225371.441172:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225371.441178:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225371.441188:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225371.441205:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7988e14 x1846777720593792/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225371.441238:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225371.441242:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225371.441261:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720593728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225371.441282:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720593792:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225371.441294:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761225371.441303:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720593664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225371.444802:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720593600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225371.445475:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720593728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225371.463175:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225371.463186:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225371.463197:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720593792:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225371.468052:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720593664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225371.505899:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225372.016061:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225372.528005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225373.040617:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225373.551989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225374.064450:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225374.575986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225375.088927:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225375.602269:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225376.113115:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225376.560126:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225376.560136:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225376.560140:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225376.560157:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 969 00000100:00080000:2.0:1761225376.560162:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225376.560169:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225376.560182:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225376.560199:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000003513afe x1846777720594048/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225376.560223:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225376.560227:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225376.560313:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720593856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225376.561029:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720594048:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225376.561316:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720593984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225376.561938:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720593920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225376.569436:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225376.569448:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225376.569458:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720594048:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225376.569567:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720593856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225376.570026:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720593920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225376.575072:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720593984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225376.623990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225377.136061:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225377.647989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225378.163651:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225378.672507:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225379.184172:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225379.696031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225380.208431:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225380.720291:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225381.232075:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225381.680743:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225381.680751:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225381.680755:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225381.680783:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 974 00000100:00080000:0.0:1761225381.680787:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225381.680792:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225381.680802:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225381.680821:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003e02fcda x1846777720594304/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225381.680849:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225381.680913:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225381.681049:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720594304:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225381.681565:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720594112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225381.681604:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720594240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225381.681627:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761225381.681632:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720594176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225381.687476:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720594176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225381.687504:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720594112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225381.687514:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720594240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225381.687637:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225381.687642:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225381.687647:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720594304:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225381.743981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225382.256081:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225382.767960:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225383.280046:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225383.791987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225384.304765:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225384.816043:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225385.329938:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225385.840029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225386.356774:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225386.800181:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225386.800189:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225386.800194:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225386.800207:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 979 00000100:00080000:2.0:1761225386.800212:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225386.800219:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225386.800232:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225386.800253:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072dbf25b x1846777720594560/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225386.800295:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225386.800299:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225386.800409:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720594560:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225386.800409:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720594368:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225386.800561:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720594496:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225386.800603:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225386.800608:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720594432:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225386.804126:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720594432:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225386.804147:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720594368:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225386.804157:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720594496:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225386.804254:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225386.804260:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225386.804265:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720594560:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225386.864022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225387.376057:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225387.891899:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225388.400076:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225388.914622:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225389.424096:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225389.936106:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225390.450693:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225390.964628:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225391.474746:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225391.920525:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225391.920533:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225391.920537:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225391.920561:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 985 00000100:00080000:0.0:1761225391.920565:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225391.920571:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225391.920582:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225391.920599:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000085a527fc x1846777720594816/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225391.920619:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225391.920623:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225391.920650:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720594688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225391.920705:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720594816:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225391.920743:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761225391.920753:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720594624:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225391.920780:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720594752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225391.923030:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720594624:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225391.923055:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720594752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225391.923067:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720594688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225391.958029:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225391.958041:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225391.958053:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720594816:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225391.984019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225392.496345:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225393.008004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225393.521553:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225394.031964:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225394.544769:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225395.056479:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225395.568275:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225396.079987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225396.592898:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225397.040329:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720594880:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225397.041817:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720594944:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225397.042899:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:2.0:1761225397.043064:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225397.043071:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225397.043075:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225397.043083:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 990 00000100:00080000:2.0:1761225397.043086:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225397.043092:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225397.043103:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225397.043123:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046bdcfa1 x1846777720595072/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225397.043177:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720595072:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225397.043822:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720595008:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225397.044128:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225397.044133:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225397.046446:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720594880:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225397.051069:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720594944:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225397.052110:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720595008:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225397.057910:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225397.057921:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225397.057930:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720595072:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225397.104244:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225397.616136:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225398.129139:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225398.640152:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225399.152014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225399.665676:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225400.175993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225400.688201:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225401.199981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225401.712144:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225402.161520:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225402.161529:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225402.161532:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225402.161544:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 995 00000100:00080000:0.0:1761225402.161548:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225402.161553:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225402.161563:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225402.161578:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a99a8bd5 x1846777720595328/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225402.161609:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225402.161613:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225402.164518:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720595136:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225402.164661:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720595264:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225402.164693:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761225402.164699:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720595200:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225402.164730:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720595328:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225402.168342:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720595136:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225402.170087:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225402.170130:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225402.170138:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720595328:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225402.170341:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720595200:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225402.170352:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720595264:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225402.223993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225402.740237:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225403.247984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225403.762986:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225404.272016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225404.784294:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225405.297232:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225405.808087:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225406.320064:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225406.832134:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225407.280123:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720595392:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225407.282241:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225407.282248:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225407.282252:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225407.282262:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1000 00000100:00080000:3.0:1761225407.282266:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225407.282271:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225407.282282:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225407.282296:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000074fca4bf x1846777720595584/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225407.282355:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720595584:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225407.283139:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720595456:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225407.283198:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225407.283205:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720595520:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225407.284037:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225407.284045:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225407.289408:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720595392:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225407.294527:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225407.294539:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225407.294550:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720595584:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225407.295236:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720595520:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225407.295250:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720595456:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225407.343986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225407.857105:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225408.371074:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225408.881812:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225409.391988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225409.904547:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225410.415980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225410.928499:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225411.440000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225411.952133:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225412.401242:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225412.401251:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225412.401256:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225412.401270:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1005 00000100:00080000:3.0:1761225412.401274:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1761225412.401279:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720595648:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225412.401282:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225412.401298:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225412.401318:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008dec2428 x1846777720595840/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225412.401348:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225412.401352:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225412.401406:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720595776:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225412.401406:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720595712:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225412.401796:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720595840:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225412.409392:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720595776:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225412.409440:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720595648:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225412.409723:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720595712:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225412.412287:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225412.412299:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225412.412309:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720595840:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225412.463946:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225412.976560:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225413.490271:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225414.002170:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225414.515187:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225415.024060:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225415.535993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225416.051501:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225416.560396:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225417.072406:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225417.520103:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225417.520114:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225417.520119:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225417.520131:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1010 00000100:00080000:3.0:1761225417.520135:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225417.520143:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225417.520155:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225417.520172:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000341ad5cd x1846777720596096/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225417.520203:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225417.520207:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225417.520342:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720595904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225417.520432:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720596032:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225417.520510:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225417.520517:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720595968:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225417.522791:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720596096:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225417.528552:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720595904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225417.532092:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720595968:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225417.532113:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720596032:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225417.534708:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225417.534720:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225417.534730:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720596096:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225417.586159:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225418.098509:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225418.607966:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225419.123027:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225419.631998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225420.144046:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225420.655982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225421.168161:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225421.680206:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225422.192065:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225422.640086:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225422.640095:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225422.640100:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225422.640113:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1015 00000100:00080000:3.0:1761225422.640117:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225422.640123:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225422.640134:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225422.640172:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8803538 x1846777720596352/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225422.640200:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225422.640203:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225422.640445:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720596352:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225422.641791:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720596160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225422.641877:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720596288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225422.641905:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225422.641909:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720596224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225422.651455:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720596288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225422.651493:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720596224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225422.651517:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720596160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225422.651590:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225422.651595:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225422.651599:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720596352:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225422.705211:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225423.216106:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225423.728369:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225424.241571:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225424.751951:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225425.265545:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225425.778388:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225426.288053:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225426.799970:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225427.312028:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225427.760106:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225427.760118:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225427.760121:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225427.760134:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1020 00000100:00080000:0.0:1761225427.760137:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225427.760143:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225427.760154:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225427.760170:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720596608/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225427.760197:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225427.760200:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225427.760222:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720596480:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225427.760310:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761225427.760316:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720596416:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225427.760364:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720596544:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225427.760597:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720596608:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225427.773218:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720596416:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225427.781413:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225427.781427:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225427.781438:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720596608:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225427.786150:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720596480:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225427.786819:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720596544:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225427.823980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225428.336077:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225428.850967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225429.360072:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225429.871983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225430.384073:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225430.895990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225431.408492:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225431.920846:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225432.432236:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225432.880823:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225432.880832:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225432.880836:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225432.880849:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1026 00000100:00080000:3.0:1761225432.880875:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225432.880885:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:2.0:1761225432.880893:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720596672:192.168.202.114@tcp:400:kworker/u8:2.0 00000008:00100000:3.0:1761225432.880897:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225432.880915:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009d0a2521 x1846777720596864/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225432.880958:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225432.880961:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225432.880989:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720596800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225432.881001:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720596736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225432.881059:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720596864:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225432.893106:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720596800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225432.893647:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225432.893654:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225432.893659:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720596864:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225432.894405:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720596736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225432.897787:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720596672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225432.944034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225433.456056:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225433.969897:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225434.480230:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225434.991947:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225435.504132:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225436.023903:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225436.532873:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225437.039980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225437.552563:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225438.000074:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225438.000084:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225438.000089:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225438.000101:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1031 00000100:00080000:2.0:1761225438.000104:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225438.000110:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225438.000123:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225438.000142:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006bca6691 x1846777720597120/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225438.000197:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225438.000200:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225438.000228:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720596992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225438.000334:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761225438.000341:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720596928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225438.000367:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720597056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225438.000600:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720597120:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225438.004203:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720596928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225438.008954:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720596992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225438.010189:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225438.010198:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225438.010206:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720597120:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225438.011915:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720597056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225438.064928:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225438.576213:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225439.091987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225439.600285:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225440.111986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225440.626949:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225441.136007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225441.649143:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225442.159984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225442.672880:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761225443.120110:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720597184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225443.122635:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720597184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225443.123295:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720597248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225443.125906:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720597248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225443.126532:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720597312:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225443.129149:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720597312:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225443.129440:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225443.129446:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225443.129451:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225443.129464:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1036 00000100:00080000:0.0:1761225443.129466:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225443.129471:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225443.129485:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225443.130446:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e956e16 x1846777720597376/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761225443.130532:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720597376:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761225443.137997:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225443.138008:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761225443.138693:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225443.138702:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225443.138712:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720597376:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225443.183987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225443.698922:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225444.207938:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225444.720251:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225445.231989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225445.744102:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225446.255983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225446.768175:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225447.283472:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225447.792096:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225448.240347:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225448.240356:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225448.240360:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225448.240371:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1041 00000100:00100000:2.0:1761225448.240372:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720597440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225448.240374:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225448.240379:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225448.240389:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225448.240404:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c3a840a0 x1846777720597632/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225448.240432:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225448.240435:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225448.240465:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720597504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225448.240466:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720597568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225448.240826:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720597632:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225448.249740:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720597440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225448.258593:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720597568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225448.258693:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720597504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225448.260997:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225448.261010:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225448.261020:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720597632:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225448.303999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225448.816170:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225449.328021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225449.843221:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225450.353657:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225450.864144:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225451.376229:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225451.888231:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225452.399993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225452.916510:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225453.361717:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720597696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225453.364983:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720597760:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225453.375184:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225453.375193:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225453.375197:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225453.375210:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1046 00000100:00080000:3.0:1761225453.375214:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225453.375221:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225453.375253:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225453.375273:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8803538 x1846777720597888/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225453.375293:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225453.375296:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225453.375359:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225453.375382:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720597824:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225453.375533:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720597888:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225453.376757:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720597760:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225453.376824:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720597696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225453.382342:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761225453.382349:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720597824:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225453.382356:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225453.382367:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720597888:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225453.426921:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225453.937718:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225454.448033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225454.960543:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225455.472009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225455.984639:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225456.496414:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225457.009974:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225457.525754:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225458.032397:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225458.480973:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225458.480981:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225458.480985:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225458.480995:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1051 00000100:00080000:2.0:1761225458.481014:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225458.481021:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225458.481030:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225458.481044:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720598144/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225458.481068:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225458.481072:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225458.481097:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720598016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225458.481244:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720598144:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225458.481349:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720597952:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225458.483218:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720598080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225458.487351:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720597952:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225458.493680:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225458.493692:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225458.493703:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720598144:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225458.495299:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720598080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225458.495411:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720598016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225458.545321:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225459.057658:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225459.569703:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225460.080908:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225460.592009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225461.104425:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225461.618929:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225462.128895:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225462.640196:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225463.152816:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225463.600188:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720598208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225463.600379:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225463.600386:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720598272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225463.601836:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225463.601844:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225463.601848:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225463.601884:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1056 00000100:00080000:3.0:1761225463.601889:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225463.601895:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225463.601909:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225463.601930:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e86c3a42 x1846777720598400/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225463.601946:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225463.601949:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225463.601984:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225463.601999:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720598336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225463.602155:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720598400:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225463.614948:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720598272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225463.614968:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720598208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225463.615610:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225463.615616:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225463.615621:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720598400:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225463.615744:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720598336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225463.664639:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225464.176088:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225464.688035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225465.210552:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225465.712030:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225466.224095:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225466.739013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225467.248153:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225467.760016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225468.273057:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225468.723410:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225468.723418:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225468.723422:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225468.723433:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1061 00000100:00080000:0.0:1761225468.723437:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225468.723495:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225468.723506:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225468.723525:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d991e6ad x1846777720598656/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225468.723555:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225468.723559:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225468.723590:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720598464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225468.723618:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720598528:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225468.723880:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720598656:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225468.723978:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720598592:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225468.738070:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720598528:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225468.738185:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720598464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225468.738211:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720598592:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225468.740243:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225468.740258:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225468.740267:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720598656:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225468.784993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225469.296128:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225469.807995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225470.321188:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225470.832010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225471.344179:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225471.856030:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225472.370079:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225472.880162:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225473.394657:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225473.840211:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225473.840220:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225473.840224:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225473.840235:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1066 00000100:00080000:2.0:1761225473.840238:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225473.840245:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225473.840255:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225473.840287:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720598912/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225473.840314:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225473.840317:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225473.840340:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720598784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225473.840466:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761225473.840473:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720598720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225473.840499:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720598848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225473.840764:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720598912:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225473.845706:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720598720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225473.845740:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720598848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225473.845752:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720598784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225473.854573:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225473.854588:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225473.854600:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720598912:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225473.904005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225474.416705:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225474.927989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225475.440105:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225475.953771:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225476.464092:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225476.977106:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225477.488086:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225477.999984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225478.512065:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225478.960245:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225478.960254:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225478.960258:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225478.960271:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1072 00000100:00080000:2.0:1761225478.960274:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225478.960282:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:3.0:1761225478.960284:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720598976:192.168.202.114@tcp:400:kworker/u8:1.0 00000008:00100000:2.0:1761225478.960294:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225478.960310:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000064ce9b52 x1846777720599168/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225478.960334:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225478.960357:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225478.960379:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720599168:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225478.960393:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599040:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225478.960400:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720599104:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225478.962941:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720599104:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225478.963057:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599040:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225478.963515:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225478.963522:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225478.963528:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720599168:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225478.965360:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720598976:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225479.025407:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225479.536339:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225480.051493:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225480.560746:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225481.071975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225481.585325:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225482.097361:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225482.609007:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225483.120006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225483.633731:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225484.082107:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720599232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225484.083176:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225484.083183:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225484.083187:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225484.083195:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1077 00000100:00080000:3.0:1761225484.083199:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225484.083218:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225484.083231:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225484.083252:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd946368 x1846777720599424/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225484.083327:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720599424:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225484.083352:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225484.083395:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225484.083402:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225484.084812:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225484.084822:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225484.090413:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225484.090448:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225484.090459:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720599232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225484.090570:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225484.090575:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225484.090579:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720599424:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225484.145580:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225484.657226:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225485.168034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225485.680103:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225486.192022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225486.705002:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225487.217687:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225487.730371:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225488.242900:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225488.754058:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225489.201754:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225489.201763:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225489.201767:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225489.201781:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1082 00000100:00080000:2.0:1761225489.201785:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225489.201792:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225489.201807:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225489.201826:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720599680/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225489.201840:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225489.201893:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225489.201927:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225489.202083:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720599680:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225489.202323:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720599488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225489.203303:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720599616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225489.207656:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225489.207716:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225489.207722:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225489.207727:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720599680:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225489.207848:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720599488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225489.207902:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720599616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225489.264392:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225489.776025:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225490.287993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225490.800157:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225491.312035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225491.825279:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225492.337727:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225492.848380:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225493.359983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225493.872365:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225494.320037:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225494.320046:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225494.320050:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225494.320060:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1087 00000100:00080000:2.0:1761225494.320063:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225494.320070:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225494.320081:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225494.320096:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a37cdda5 x1846777720599936/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225494.320120:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225494.320123:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225494.320177:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720599744:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225494.320537:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599872:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225494.320580:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225494.320587:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599808:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225494.320959:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720599936:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225494.328353:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720599744:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225494.328391:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599872:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:02020000:0.0:1761225494.344159:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00100000:2.0:1761225494.345785:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720599808:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225494.352921:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225494.352943:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225494.352958:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720599936:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225494.384078:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225494.896071:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225495.407981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225495.920075:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225496.432115:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225496.944923:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225497.455980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225497.968935:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225498.481121:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225498.992651:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225499.440064:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225499.440072:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225499.440076:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225499.440087:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1092 00000100:00080000:2.0:1761225499.440090:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225499.440097:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225499.440107:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225499.440121:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f5134a53 x1846777720600192/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225499.440204:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720600192:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761225499.441200:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225499.441207:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225499.441231:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720600064:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225499.441301:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761225499.441307:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720600000:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225499.441329:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720600128:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225499.443074:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225499.443085:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225499.443094:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720600192:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225499.444243:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720600000:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225499.444261:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720600128:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225499.444271:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720600064:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225499.510415:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225500.016074:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225500.530068:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225501.040093:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225501.551984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225502.064051:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225502.578032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225503.090847:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225503.600022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225504.112105:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225504.560821:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225504.560830:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225504.560835:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225504.560846:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1097 00000100:00080000:3.0:1761225504.560850:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225504.560914:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225504.560926:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225504.560944:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720600448/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225504.560959:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225504.560963:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225504.560992:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720600320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225504.561109:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761225504.561117:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720600256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225504.561154:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720600384:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225504.562072:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720600448:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225504.563063:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720600256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225504.563084:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720600384:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225504.563097:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720600320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225504.565349:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225504.565361:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225504.565371:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720600448:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225504.626506:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225505.136163:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225505.647979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225506.166506:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225506.678410:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225507.184051:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225507.695985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225508.208075:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225508.719983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225509.232104:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225509.680907:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225509.680915:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225509.680919:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225509.680929:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1102 00000100:00080000:2.0:1761225509.680932:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225509.680938:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225509.680947:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225509.680962:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f5134a53 x1846777720600704/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225509.680978:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225509.680981:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225509.681043:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720600704:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225509.681443:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720600512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225509.681478:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720600640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225509.681496:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225509.681500:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720600576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225509.687012:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225509.687025:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225509.687034:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720600704:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225509.687395:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720600512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225509.689392:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720600576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225509.689412:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720600640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225509.744003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225510.258147:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225510.767985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225511.280662:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225511.792529:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225512.305109:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225512.817525:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225513.331994:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225513.840606:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225514.352251:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225514.800043:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225514.800052:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225514.800071:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225514.800082:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1107 00000100:00080000:2.0:1761225514.800086:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225514.800092:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225514.800101:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225514.800116:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720600960/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225514.800195:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720600960:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225514.801589:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720600768:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225514.801655:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720600896:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225514.801697:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225514.801704:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720600832:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225514.803490:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225514.803500:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225514.811377:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720600832:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225514.811424:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720600768:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225514.811436:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720600896:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225514.814295:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225514.814305:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225514.814314:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720600960:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225514.863995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225515.377092:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225515.888317:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225516.400041:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225516.912545:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225517.427823:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225517.936274:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225518.457552:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225518.960010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225519.473689:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225519.920047:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225519.920055:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225519.920060:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225519.920070:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1113 00000100:00080000:2.0:1761225519.920074:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225519.920080:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225519.920091:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225519.920102:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720601024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225519.920107:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720601216/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225519.920134:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225519.920138:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225519.920190:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720601088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225519.920207:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720601152:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225519.921192:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720601216:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225519.923903:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720601024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225519.923921:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720601152:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225519.924018:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720601088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225519.934135:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225519.934146:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225519.934155:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720601216:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225519.984581:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225520.496117:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225521.008328:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225521.522014:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225522.032534:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225522.544161:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225523.055969:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225523.568463:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225524.080021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225524.593110:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225525.040119:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225525.040127:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225525.040131:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1761225525.040138:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720601280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225525.040140:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1118 00000100:00080000:3.0:1761225525.040145:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225525.040157:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225525.040172:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225525.040206:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b01aecf x1846777720601472/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225525.040221:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225525.040224:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225525.040237:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720601408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225525.040270:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225525.040275:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720601344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225525.040341:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720601472:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761225525.043242:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225525.043252:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225525.043261:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720601472:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225525.043306:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720601344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225525.043321:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720601280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225525.043331:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720601408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225525.104493:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225525.618032:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225526.128610:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225526.640390:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225527.155434:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225527.664335:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225528.175980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225528.688930:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225529.200636:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225529.714890:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225530.160100:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225530.160108:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225530.160112:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225530.160123:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1123 00000100:00080000:0.0:1761225530.160127:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225530.160133:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225530.160144:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225530.160161:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b0db11ae x1846777720601728/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225530.160199:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225530.160203:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225530.160254:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720601536:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225530.160281:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720601600:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225530.160416:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720601664:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225530.161520:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720601728:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225530.164002:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720601600:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225530.164047:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720601536:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225530.164058:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720601664:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225530.168515:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225530.168529:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225530.168539:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720601728:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225530.223983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225530.736154:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225531.250704:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225531.760631:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225532.271997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225532.788113:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225533.295993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225533.808111:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225534.319980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225534.832348:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761225535.280111:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720601792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225535.281587:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720601856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225535.282181:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761225535.282205:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720601920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225535.282953:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225535.282962:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225535.282966:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225535.282973:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1128 00000100:00080000:0.0:1761225535.282976:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225535.282982:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225535.282993:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225535.283015:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720601984/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761225535.283083:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720601984:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761225535.284056:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225535.284063:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225535.287762:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720601920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225535.287805:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720601856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225535.288415:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720601792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225535.289416:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225535.289425:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225535.289444:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720601984:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225535.343981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225535.860596:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225536.367991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225536.880090:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225537.400416:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225537.905116:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225538.416001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225538.928073:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225539.440001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225539.952057:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225540.403084:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225540.403092:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225540.403096:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225540.403107:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1133 00000100:00080000:0.0:1761225540.403110:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225540.403124:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225540.403133:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225540.403156:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fe9fa97 x1846777720602240/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225540.403189:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225540.403192:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225540.403224:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720602112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225540.403332:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761225540.403338:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720602048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225540.403410:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720602176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225540.403530:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720602240:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761225540.406465:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225540.406475:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225540.406485:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720602240:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225540.418522:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720602048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225540.418550:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720602176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225540.418563:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720602112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225540.465824:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225540.977519:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225541.489524:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225542.000093:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225542.512227:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225543.024073:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225543.536216:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225544.048270:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225544.559970:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225545.073929:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225545.520073:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225545.520082:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225545.520085:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225545.520096:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1138 00000100:00080000:0.0:1761225545.520100:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225545.520106:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225545.520117:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225545.520132:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720602496/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225545.520159:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225545.520162:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225545.520469:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720602368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225545.520568:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761225545.520574:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720602304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225545.520598:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720602432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225545.520748:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720602496:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225545.523544:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720602304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225545.523567:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720602432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225545.523578:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720602368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225545.523819:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225545.523824:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225545.523829:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720602496:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225545.584929:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225546.096038:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225546.607989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225547.120179:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225547.633030:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225548.147905:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225548.658602:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225549.168747:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225549.681257:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225550.192088:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225550.640124:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720602560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225550.640949:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761225550.640961:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720602624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225550.641666:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225550.641673:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225550.641676:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225550.641682:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1143 00000100:00080000:1.0:1761225550.641685:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225550.641691:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225550.641700:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225550.641714:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720602752/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225550.641738:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225550.641742:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225550.641766:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761225550.641778:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720602688:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225550.641785:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720602752:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225550.643922:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720602624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225550.643944:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720602560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225550.645321:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225550.645329:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225550.645336:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720602752:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225550.646130:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720602688:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225550.710347:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225551.216052:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225551.728297:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225552.240089:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225552.751951:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225553.265884:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225553.776046:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225554.288182:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225554.799991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225555.312114:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225555.760472:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225555.760481:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225555.760486:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225555.760498:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1148 00000100:00100000:0.0:1761225555.760500:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720602816:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225555.760519:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225555.760527:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225555.760538:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225555.760556:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005d7eada1 x1846777720603008/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225555.760584:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225555.760588:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225555.760594:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720602944:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225555.760623:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720602880:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225555.760775:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720603008:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225555.764160:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720602816:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225555.766289:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720602944:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225555.766353:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720602880:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225555.766516:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225555.766536:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225555.766541:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720603008:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225555.823979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225556.336126:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225556.847982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225557.361672:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225557.872476:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225558.388248:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225558.896286:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225559.409312:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225559.920022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225560.434756:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225560.880088:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720603072:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225560.883449:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225560.883456:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225560.883460:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225560.883469:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1154 00000100:00080000:1.0:1761225560.883472:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225560.883478:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225560.883487:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225560.883501:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017ee9697 x1846777720603264/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225560.883521:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225560.883524:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225560.883597:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720603200:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225560.883772:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761225560.883779:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720603136:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225560.883892:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720603264:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225560.885591:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720603200:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225560.895318:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225560.895333:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225560.895342:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720603264:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225560.895424:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720603136:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225560.895434:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720603072:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225560.947290:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225561.456272:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225561.968151:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225562.480203:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225562.993801:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225563.507185:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225564.017457:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225564.530174:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225565.039997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225565.554249:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225566.000041:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225566.000048:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225566.000052:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225566.000063:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1159 00000100:00080000:1.0:1761225566.000066:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225566.000071:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225566.000081:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225566.000099:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aa6e6fe7 x1846777720603520/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225566.000107:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720603328:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225566.000151:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225566.000155:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225566.000173:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720603520:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225566.000204:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720603456:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225566.000239:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720603392:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225566.005746:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225566.005756:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225566.005764:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720603520:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225566.007545:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720603392:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225566.009010:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720603328:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225566.009030:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720603456:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225566.063979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225566.576793:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225567.089295:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225567.600496:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225568.112134:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225568.627257:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225569.136725:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225569.651396:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225570.160259:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225570.672018:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225571.120281:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720603584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225571.121130:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225571.121136:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225571.121139:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225571.121146:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1164 00000100:00080000:2.0:1761225571.121148:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225571.121154:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225571.121165:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225571.121179:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003ba27032 x1846777720603776/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225571.121208:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225571.121211:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225571.121232:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720603648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225571.121235:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720603776:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225571.121280:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225571.121286:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720603712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225571.122263:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720603584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225571.124676:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225571.124687:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225571.124700:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720603776:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225571.130465:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720603712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225571.130506:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720603648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225571.183979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225571.700180:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225572.208561:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225572.722378:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225573.231995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225573.744536:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225574.256265:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225574.768833:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225575.279991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225575.793204:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225576.240167:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225576.240173:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225576.240177:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225576.240187:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1169 00000100:00080000:1.0:1761225576.240190:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225576.240195:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225576.240204:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225576.240216:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b1564d60 x1846777720604032/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225576.240236:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225576.240238:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225576.240262:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720603904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225576.240313:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720603840:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225576.240358:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720603968:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225576.240416:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720604032:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225576.245324:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720603968:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225576.249181:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720603904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225576.249339:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225576.249345:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225576.249349:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720604032:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225576.250917:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720603840:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225576.304972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225576.817372:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225577.327986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225577.840331:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225578.353732:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225578.865665:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225579.379771:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225579.888057:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225580.400041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225580.912121:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225581.360178:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225581.360205:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720604096:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225581.360344:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720604160:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225581.360394:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225581.360400:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225581.360403:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225581.360408:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1174 00000100:00080000:2.0:1761225581.360411:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225581.360416:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:2.0:1761225581.360423:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225581.360429:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720604224:192.168.202.114@tcp:400:kworker/u8:1.0 00000008:00100000:2.0:1761225581.360463:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225581.360482:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046bdcfa1 x1846777720604288/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761225581.360565:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720604288:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761225581.361129:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225581.361134:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225581.368898:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720604224:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225581.368920:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720604160:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225581.370409:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225581.370439:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225581.370448:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720604288:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225581.371955:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720604096:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225581.424409:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225581.937310:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225582.449542:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225582.960175:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225583.472373:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225583.986364:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225584.495998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225585.009066:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225585.520015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225586.032444:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225586.480507:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225586.480516:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225586.480520:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225586.480530:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1179 00000100:00080000:2.0:1761225586.480534:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225586.480541:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225586.480552:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225586.480567:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001379e0d8 x1846777720604544/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225586.480592:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225586.480595:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225586.480924:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720604352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225586.482030:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720604544:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225586.483714:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720604480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225586.483807:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225586.483817:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720604416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225586.485307:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720604352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225586.486549:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720604480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225586.486572:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720604416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225586.486662:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225586.486667:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225586.486672:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720604544:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225586.544270:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225587.056693:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225587.567993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225588.080276:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225588.591988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225589.104050:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225589.617589:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225590.129065:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225590.640192:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225591.152070:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225591.600223:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225591.600231:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225591.600250:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225591.600261:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1184 00000100:00080000:0.0:1761225591.600264:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225591.600271:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225591.600282:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225591.600299:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000243e8b76 x1846777720604800/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225591.600330:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225591.600334:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225591.600360:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720604672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225591.600445:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761225591.600451:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720604608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225591.600472:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720604736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225591.600820:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720604800:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761225591.606330:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1761225591.606336:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720604736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225591.606341:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225591.606350:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720604800:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225591.606356:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720604672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225591.611587:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720604608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225591.664168:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225592.177279:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225592.688110:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225593.200488:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225593.712004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225594.224115:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225594.735996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225595.247979:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225595.759983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225596.272609:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225596.720099:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225596.720107:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225596.720111:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225596.720121:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1189 00000100:00080000:0.0:1761225596.720125:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225596.720131:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225596.720141:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225596.720159:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000243e8b76 x1846777720605056/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225596.720188:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225596.720209:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225596.720232:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720604928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225596.720329:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761225596.720336:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720604864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225596.720361:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720604992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225596.721101:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720605056:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225596.724268:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720604864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225596.724294:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720604928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225596.725565:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720604992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225596.727305:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225596.727315:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225596.727323:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720605056:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225596.784016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225597.298897:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225597.807998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225598.320084:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225598.832570:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225599.345284:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225599.856379:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225600.368367:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225600.880048:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225601.392225:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225601.840125:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225601.840132:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225601.840136:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225601.840148:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1194 00000100:00080000:3.0:1761225601.840166:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225601.840173:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225601.840184:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225601.840202:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003d27cd56 x1846777720605312/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761225601.840280:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720605312:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:3.0:1761225601.840400:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761225601.840405:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720605120:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225601.840406:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225601.840434:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720605184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225601.840450:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720605248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225601.851469:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225601.851485:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225601.851496:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720605312:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225601.851667:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720605184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225601.851825:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720605120:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225601.851840:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720605248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225601.904012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225602.416106:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225602.927981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225603.442564:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225603.951991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225604.464062:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225604.976311:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225605.489506:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225606.002259:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225606.512186:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225606.960186:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720605376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225606.960343:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225606.960350:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720605440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225606.960582:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225606.960588:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225606.960591:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225606.960597:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1200 00000100:00080000:1.0:1761225606.960600:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225606.960605:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225606.960615:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225606.960631:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d0d44bb x1846777720605568/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225606.960678:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720605504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225606.960728:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720605568:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761225606.960938:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225606.960943:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761225606.963207:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225606.963217:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225606.963221:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720605504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225606.963225:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720605568:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225606.963241:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720605440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225606.963257:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720605376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225607.023995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225607.536801:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225608.047988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225608.560266:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225609.071991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225609.586474:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225610.096380:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225610.609633:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225611.121364:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225611.632176:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225612.080120:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225612.080130:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225612.080134:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225612.080146:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1205 00000100:00080000:3.0:1761225612.080150:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225612.080156:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:2.0:1761225612.080252:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720605632:192.168.202.114@tcp:400:kworker/u8:1.0 00000008:00100000:3.0:1761225612.080268:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225612.080291:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002b7677b0 x1846777720605824/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225612.080319:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225612.080323:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225612.080344:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720605696:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225612.080999:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720605824:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225612.081494:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720605760:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225612.088081:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225612.088095:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225612.088104:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720605824:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225612.088598:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720605696:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225612.088658:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720605760:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225612.088675:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720605632:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225612.144012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225612.658441:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225613.167969:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225613.680116:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225614.191977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225614.704307:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225615.218388:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225615.728071:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225616.239988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225616.752940:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225617.200104:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225617.200112:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225617.200116:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761225617.200120:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720605888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225617.200125:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1210 00000100:00080000:2.0:1761225617.200130:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225617.200137:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225617.200149:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225617.200165:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c9f23bda x1846777720606080/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761225617.200235:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720606080:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225617.200521:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720606016:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225617.200781:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225617.200788:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720605952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225617.201120:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225617.201125:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225617.206728:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720605952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225617.206744:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720605888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225617.206754:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720606016:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225617.206791:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225617.206797:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225617.206803:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720606080:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225617.264143:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225617.776124:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225618.288016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225618.800577:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225619.312534:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225619.824166:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225620.335948:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225620.849126:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225621.362920:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225621.873847:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225622.322309:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225622.322317:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225622.322321:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225622.322331:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1215 00000100:00080000:0.0:1761225622.322334:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225622.322340:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225622.322368:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225622.322385:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fe9fa97 x1846777720606336/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225622.322411:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225622.322415:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225622.322477:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720606336:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225622.322795:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720606208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225622.322845:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761225622.322884:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720606144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225622.322911:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720606272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225622.329328:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225622.329345:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225622.329355:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720606336:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225622.329388:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720606144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225622.329420:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720606272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225622.329430:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720606208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225622.384590:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225622.897133:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225623.408195:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225623.920090:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225624.431982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225624.944076:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225625.455994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225625.968050:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225626.479996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225626.993751:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225627.440296:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720606400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225627.440736:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225627.440743:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720606464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225627.440929:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225627.440935:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225627.440939:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225627.440946:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1220 00000100:00080000:3.0:1761225627.440949:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225627.440957:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225627.440971:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225627.440993:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000727a12f1 x1846777720606592/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225627.441010:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225627.441031:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225627.441050:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225627.441059:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720606528:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225627.441167:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720606592:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225627.447042:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720606400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225627.447468:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720606528:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225627.447751:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225627.447757:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225627.447763:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720606592:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225627.447805:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720606464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225627.503982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225628.016177:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225628.528037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225629.041528:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225629.552001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225630.075080:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225630.576284:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225631.092524:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225631.604312:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225632.112103:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225632.560103:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225632.560112:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225632.560117:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225632.560130:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1225 00000100:00080000:3.0:1761225632.560133:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225632.560141:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225632.560151:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225632.560168:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000074fca4bf x1846777720606848/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761225632.560249:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720606720:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225632.560254:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720606848:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225632.560436:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761225632.560454:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720606656:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225632.560490:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720606784:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225632.567216:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225632.567229:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761225632.567832:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225632.567839:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225632.567849:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720606848:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225632.568187:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720606656:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225632.568201:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720606784:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225632.568220:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720606720:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225632.624023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225633.136656:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225633.648009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225634.162329:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225634.672131:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225635.188055:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225635.695977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225636.209398:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225636.721845:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225637.232203:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225637.684934:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225637.684942:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225637.684946:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225637.684957:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1230 00000100:00080000:2.0:1761225637.684960:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225637.684967:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225637.684977:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225637.684992:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072dbf25b x1846777720607104/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225637.685020:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225637.685023:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225637.685050:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720606976:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225637.685159:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761225637.685166:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720606912:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225637.685191:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720607040:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225637.688049:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720607104:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761225637.696534:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225637.696548:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225637.696558:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720607104:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225637.696758:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720606912:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225637.696774:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720607040:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225637.696786:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720606976:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225637.743948:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225638.259046:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225638.767989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225639.280264:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225639.792089:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225640.304709:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225640.816094:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225641.329194:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225641.840063:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225642.357489:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225642.802962:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225642.802970:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225642.802975:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761225642.802981:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720607168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225642.802985:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1235 00000100:00080000:0.0:1761225642.802990:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225642.802996:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225642.803007:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225642.803025:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720607360/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225642.803055:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225642.803060:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225642.803065:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720607296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225642.803095:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225642.803097:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720607360:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225642.803104:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720607232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225642.807998:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720607232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225642.808003:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225642.808010:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225642.808015:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720607360:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225642.808023:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720607296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225642.809045:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720607168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225642.863982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225643.377137:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225643.888635:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225644.400072:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225644.913125:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225645.425398:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225645.938581:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225646.448391:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225646.961954:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225647.472062:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225647.920125:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225647.920134:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225647.920138:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761225647.920141:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720607488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225647.920144:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1241 00000100:00080000:0.0:1761225647.920164:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225647.920171:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225647.920181:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225647.920196:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720607616/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225647.920216:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225647.920219:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225647.920236:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761225647.920243:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720607424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225647.920278:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720607552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225647.920677:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720607616:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225647.934233:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720607488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225647.937626:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720607424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225647.942031:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720607552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:3.0:1761225647.944064:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:3.0:1761225647.949053:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225647.949066:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225647.949077:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720607616:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225647.986141:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225648.496145:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225649.007983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225649.521113:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225650.031992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225650.544131:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225651.056218:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225651.569116:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225652.083718:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225652.592060:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225653.040084:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225653.040092:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225653.040096:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225653.040106:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1246 00000100:00080000:0.0:1761225653.040109:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225653.040115:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225653.040124:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225653.040137:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720607872/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225653.040160:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225653.040164:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225653.040188:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720607744:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225653.040273:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761225653.040278:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720607680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225653.040296:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720607808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225653.040544:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720607872:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225653.045055:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720607744:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225653.050304:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225653.050317:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225653.050327:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720607872:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225653.050708:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720607680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225653.050724:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720607808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225653.105339:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225653.616075:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225654.127996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225654.640167:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225655.152162:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225655.664056:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225656.175934:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225656.688057:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225657.201128:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225657.715479:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225658.160090:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225658.160099:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225658.160103:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225658.160116:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1251 00000100:00080000:3.0:1761225658.160120:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225658.160126:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225658.160138:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225658.160153:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000006d85789 x1846777720608128/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225658.160182:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225658.160185:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225658.160210:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720608000:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225658.160236:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720608128:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225658.160744:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720607936:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225658.163535:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720608064:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225658.170566:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225658.170578:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225658.170587:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720608128:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225658.174380:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720608000:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225658.177121:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720607936:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225658.177145:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720608064:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225658.223994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225658.736411:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225659.248371:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225659.760430:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225660.272879:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225660.788358:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225661.296887:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225661.809745:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225662.319980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225662.834175:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225663.283724:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225663.283733:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225663.283737:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225663.283749:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1256 00000100:00080000:1.0:1761225663.283752:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225663.283758:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225663.283769:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225663.283802:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a2fc13a x1846777720608384/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225663.283832:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225663.283835:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225663.283906:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720608256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225663.284001:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761225663.284007:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720608192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225663.284034:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720608320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225663.285037:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720608384:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225663.288904:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720608192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225663.288929:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720608320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225663.288941:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720608256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225663.294377:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225663.294389:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225663.294399:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720608384:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225663.344845:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225663.859723:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225664.367988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225664.880595:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225665.391975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225665.905232:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225666.416744:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225666.928050:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225667.439982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225667.952073:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225668.400056:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225668.400064:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225668.400068:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225668.400080:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1261 00000100:00080000:1.0:1761225668.400084:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225668.400090:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225668.400101:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225668.400117:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094e95ca3 x1846777720608640/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225668.400156:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225668.400160:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225668.400289:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720608512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225668.400400:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761225668.400407:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720608448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225668.400437:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720608576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225668.402576:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720608640:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225668.408242:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720608448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225668.408262:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720608576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225668.408271:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720608512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225668.414798:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225668.414808:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225668.414819:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720608640:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225668.464808:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225668.976301:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225669.488256:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225670.001126:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225670.511990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225671.024055:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225671.535987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225672.048078:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225672.559985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225673.072141:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225673.520129:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225673.520137:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225673.520141:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225673.520151:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1266 00000100:00080000:3.0:1761225673.520168:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225673.520182:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225673.520191:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225673.520206:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720608896/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225673.520229:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225673.520233:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225673.520316:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720608896:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225673.520978:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720608704:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225673.521562:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720608832:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225673.521604:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225673.521611:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720608768:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225673.526341:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225673.526353:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225673.526363:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720608896:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225673.526453:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720608768:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225673.526465:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720608704:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225673.526475:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720608832:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225673.585055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225674.098883:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225674.607977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225675.120582:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225675.632061:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225676.144413:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225676.658002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225677.170365:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225677.679987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225678.192088:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225678.640176:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720608960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225678.640625:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225678.640633:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225678.641699:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225678.641707:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225678.641711:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225678.641720:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1271 00000100:00080000:2.0:1761225678.641724:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225678.641730:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225678.641741:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225678.641765:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c987228a x1846777720609152/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225678.641808:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225678.641812:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225678.641828:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225678.641840:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720609152:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225678.641843:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720609088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225678.674104:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720609088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225678.674161:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225678.674175:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720608960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225678.674304:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225678.674309:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225678.674314:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720609152:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225678.704572:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225679.216603:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225679.731516:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225680.240057:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225680.751968:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225681.264093:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225681.775920:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225682.290990:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225682.800739:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225683.312075:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225683.760194:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225683.760203:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225683.760207:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225683.760218:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1276 00000100:00080000:3.0:1761225683.760221:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225683.760227:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225683.760237:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225683.760252:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a6d0bfc0 x1846777720609408/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225683.760275:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225683.760278:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225683.760301:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225683.760403:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761225683.760409:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720609216:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225683.760430:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225683.760499:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720609408:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761225683.765702:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225683.765714:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225683.765724:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720609408:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225683.765911:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720609216:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225683.765929:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225683.765941:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225683.824335:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225684.340068:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225684.848006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225685.360107:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225685.871987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225686.389031:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225686.897952:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225687.408171:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225687.919979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225688.435143:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225688.880096:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720609472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225688.880368:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225688.880376:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225688.880601:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225688.880623:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225688.880627:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225688.880632:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1282 00000100:00080000:3.0:1761225688.880636:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225688.880641:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225688.880651:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225688.880667:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e86c3a42 x1846777720609664/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225688.880690:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225688.880693:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225688.880706:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225688.880714:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720609600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225688.880720:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720609664:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761225688.884518:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225688.884530:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225688.884540:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720609664:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225688.884608:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225688.884620:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720609472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225688.884904:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720609600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225688.943988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225689.456108:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225689.969438:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225690.483344:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225690.992130:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225691.504485:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225692.015987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225692.528812:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225693.041178:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225693.552046:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225694.000151:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720609728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225694.001340:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225694.002651:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720609856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225694.004383:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225694.004394:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225694.004399:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225694.004409:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1287 00000100:00080000:3.0:1761225694.004412:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225694.004418:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225694.004430:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225694.004474:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8803538 x1846777720609920/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225694.004551:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720609920:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761225694.005586:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225694.005594:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225694.010388:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720609728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225694.011111:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720609792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225694.011195:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225694.011201:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225694.011205:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720609920:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225694.011266:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720609856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225694.063977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225694.576196:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225695.087976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225695.600322:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225696.112126:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225696.625338:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225697.136330:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225697.648093:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225698.160006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225698.673221:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225699.122067:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225699.122076:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225699.122079:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761225699.122224:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720609984:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225699.122308:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1292 00000100:00080000:1.0:1761225699.122311:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225699.122317:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225699.122328:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225699.122347:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094e95ca3 x1846777720610176/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225699.122371:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225699.122375:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225699.122409:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720610048:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225699.122834:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720610176:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225699.123439:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720610112:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225699.135083:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225699.135095:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225699.135104:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720610176:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225699.135497:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720610048:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225699.135529:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720609984:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225699.135539:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720610112:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225699.184003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225699.696356:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225700.208005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225700.722555:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225701.232733:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225701.745013:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225702.256820:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225702.768059:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225703.286341:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225703.793810:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225704.240995:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225704.241005:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225704.241009:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225704.241021:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1297 00000100:00100000:3.0:1761225704.241021:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720610240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225704.241024:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225704.241031:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225704.241041:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225704.241055:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720610432/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225704.241078:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225704.241099:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225704.241111:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720610432:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225704.243960:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720610304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225704.246447:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720610368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225704.250334:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225704.250346:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225704.250356:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720610432:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225704.257461:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720610304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225704.257531:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720610240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225704.257542:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720610368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225704.303998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225704.816181:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225705.328032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225705.840208:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225706.352019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225706.864273:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225707.376006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225707.888466:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225708.399999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225708.912140:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225709.360069:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225709.360078:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225709.360083:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225709.360095:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1302 00000100:00080000:1.0:1761225709.360099:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225709.360106:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:0.0:1761225709.360109:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720610496:192.168.202.114@tcp:400:kworker/u8:1.0 00000008:00100000:1.0:1761225709.360118:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225709.360134:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a2fc13a x1846777720610688/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225709.360161:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225709.360164:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225709.360196:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720610560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225709.360222:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720610624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225709.361748:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720610688:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225709.368399:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720610560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225709.370171:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720610496:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225709.370187:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720610624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225709.370483:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225709.370489:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225709.370493:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720610688:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225709.423981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225709.936672:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225710.448015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225710.960077:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225711.471981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225711.984107:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225712.495996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225713.008150:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225713.520239:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225714.034160:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225714.481377:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225714.481386:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225714.481390:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225714.481418:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1307 00000100:00080000:1.0:1761225714.481422:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225714.481428:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225714.481439:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225714.481456:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b1564d60 x1846777720610944/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225714.481484:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225714.481487:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225714.481534:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720610944:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225714.481547:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720610752:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225714.481629:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720610880:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225714.481662:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761225714.481668:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720610816:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225714.489508:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225714.489520:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225714.489530:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720610944:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225714.490503:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720610752:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225714.490520:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720610880:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225714.490979:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720610816:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225714.546882:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225715.056440:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225715.568889:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225716.080069:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225716.591984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225717.107397:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225717.615981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225718.128009:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225718.639987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225719.157364:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225719.600123:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225719.600131:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225719.600135:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225719.600146:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1312 00000100:00080000:2.0:1761225719.600150:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225719.600156:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225719.600166:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225719.600181:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003ba27032 x1846777720611200/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225719.600209:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225719.600229:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225719.600323:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720611200:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225719.600333:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720611008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225719.600620:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720611136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225719.600654:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225719.600659:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720611072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225719.602943:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720611008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225719.603063:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225719.603067:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225719.603072:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720611200:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225719.606000:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720611136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225719.610632:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720611072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225719.664400:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225720.176834:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225720.689323:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225721.203397:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225721.714716:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225722.224198:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225722.736254:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225723.248801:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225723.759901:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225724.272231:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225724.720318:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225724.720327:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225724.720332:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225724.720344:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1317 00000100:00080000:2.0:1761225724.720349:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:3.0:1761225724.720350:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720611264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225724.720358:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225724.720369:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225724.720386:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000790fc921 x1846777720611456/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225724.720460:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720611456:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225724.720462:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720611392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225724.720504:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225724.720510:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720611328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761225724.720996:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225724.721001:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225724.726610:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720611328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225724.726891:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225724.726896:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225724.726901:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720611456:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225724.726977:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720611264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225724.726993:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720611392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225724.784121:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225725.296027:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225725.808037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225726.320065:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225726.832409:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225727.344263:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225727.857038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225728.368008:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225728.879983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225729.395114:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225729.840120:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225729.840129:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225729.840133:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225729.840144:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1322 00000100:00080000:1.0:1761225729.840147:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225729.840153:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225729.840179:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225729.840197:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720611712/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225729.840228:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225729.840232:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225729.840314:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720611712:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225729.840483:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720611520:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225729.840517:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720611648:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225729.840551:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225729.840557:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720611584:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225729.843724:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720611584:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225729.843744:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720611648:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225729.845990:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720611520:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225729.846085:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225729.846090:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225729.846094:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720611712:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225729.903970:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225730.419408:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225730.929761:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225731.440062:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225731.951972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225732.464625:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225732.976054:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225733.488066:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225733.999933:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225734.512589:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225734.975800:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225734.975808:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225734.975811:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225734.975822:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1328 00000100:00080000:2.0:1761225734.975826:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225734.975832:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225734.975841:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225734.975898:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec5c4aea x1846777720611968/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225734.975913:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225734.975916:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225734.976671:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720611776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225734.976754:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720611904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225734.976777:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225734.976781:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720611840:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225734.977162:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720611968:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225734.980255:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720611840:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225734.980275:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720611776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225734.980284:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720611904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225734.980486:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225734.980491:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225734.980495:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720611968:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225735.024013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225735.536412:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225736.047980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225736.560440:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225737.072326:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225737.584121:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225738.095967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225738.608049:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225739.119981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225739.632181:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225740.082109:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225740.082116:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225740.082120:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1761225740.082124:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720612032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225740.082126:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1333 00000100:00080000:3.0:1761225740.082129:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225740.082135:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225740.082144:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225740.082156:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f9712cd6 x1846777720612224/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225740.082178:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225740.082182:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225740.082198:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225740.082204:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720612160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225740.082285:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720612224:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225740.091780:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720612032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225740.094294:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720612160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225740.094369:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225740.097730:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225740.097740:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225740.097749:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720612224:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225740.144046:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225740.656620:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225741.167967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225741.680821:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225742.193833:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225742.704208:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225743.216012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225743.731209:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225744.240083:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225744.753571:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225745.200431:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225745.200439:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225745.200443:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225745.200453:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1338 00000100:00080000:3.0:1761225745.200457:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1761225745.200462:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225745.200471:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225745.200484:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c3a840a0 x1846777720612480/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225745.200506:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225745.200509:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225745.200717:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720612288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225745.200744:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720612480:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225745.201201:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225745.201231:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225745.201237:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225745.213425:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225745.213442:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720612288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225745.213452:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225745.213548:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225745.213562:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225745.213566:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720612480:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225745.264609:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225745.776044:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225746.288005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225746.802692:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225747.315113:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225747.824879:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225748.335987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225748.848497:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225749.360451:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225749.872216:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225750.322431:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225750.322439:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225750.322443:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225750.322455:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1343 00000100:00080000:2.0:1761225750.322460:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1761225750.322466:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225750.322477:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225750.322494:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f5134a53 x1846777720612736/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225750.322528:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225750.322551:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225750.322585:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225750.322709:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761225750.322716:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720612544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225750.322742:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225750.322982:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720612736:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761225750.327343:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225750.327354:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225750.327379:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720612736:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225750.327776:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720612544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225750.327793:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225750.327804:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225750.385275:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225750.896714:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225751.408003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225751.929976:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225752.431976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225752.953337:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225753.456246:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225753.968124:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225754.480072:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225754.992237:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225755.443348:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225755.443356:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225755.443360:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225755.443371:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1348 00000100:00080000:2.0:1761225755.443375:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 16s 00000100:00080000:2.0:1761225755.443380:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225755.443391:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225755.443407:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f5134a53 x1846777720612992/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225755.443455:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225755.443458:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225755.443494:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720612864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225755.443514:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720612992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225755.443614:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720612800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225755.446320:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225755.449253:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225755.449264:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225755.449273:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720612992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225755.453007:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720612864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225755.453140:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720612800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225755.453153:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720612928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225755.503980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225756.016513:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225756.527982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225757.040580:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225757.551982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225758.063990:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225758.576240:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225759.088534:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225759.599983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225760.112729:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225760.560556:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225760.560566:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225760.560570:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225760.560584:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1353 00000100:00080000:3.0:1761225760.560589:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761225760.560596:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225760.560607:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225760.560625:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd946368 x1846777720613248/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225760.560704:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720613248:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225760.560850:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720613056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225760.560909:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720613184:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225760.560937:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225760.560943:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720613120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225760.561331:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225760.561337:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225760.571777:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720613120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225760.571794:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720613184:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225760.578513:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720613056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225760.579475:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225760.579482:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225760.579488:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720613248:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225760.624003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225761.136391:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225761.648171:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225762.159996:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225762.672894:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225763.184404:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225763.698901:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225764.209326:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225764.719990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225765.234986:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225765.680069:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225765.680077:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225765.680082:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225765.680094:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1358 00000100:00080000:3.0:1761225765.680097:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761225765.680104:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225765.680114:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225765.680129:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a6d0bfc0 x1846777720613504/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225765.680156:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225765.680159:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225765.680203:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720613504:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225765.680299:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720613312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225765.680334:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720613440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225765.680357:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225765.680362:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720613376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225765.683168:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225765.683180:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225765.683191:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720613504:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761225765.690401:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720613312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225765.690835:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720613376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225765.690846:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720613440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225765.748069:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225766.256082:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225766.767984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225767.280177:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225767.792662:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225768.304203:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225768.815935:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225769.329531:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225769.843707:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225770.353117:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225770.800141:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225770.800149:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225770.800153:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225770.800166:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1363 00000100:00080000:0.0:1761225770.800170:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761225770.800178:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225770.800208:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225770.800227:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003e02fcda x1846777720613760/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225770.800250:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225770.800253:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225770.800316:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720613760:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225770.800519:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720613568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225770.800553:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720613696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225770.800573:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761225770.800578:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720613632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225770.806002:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720613696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225770.806409:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225770.806415:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225770.806421:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720613760:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225770.806493:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720613568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225770.806806:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720613632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761225770.864021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225771.376101:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225771.889276:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225772.402188:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225772.912690:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225773.425542:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225773.939438:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225774.448106:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225774.961684:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225775.472477:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225775.920113:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225775.920122:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225775.920126:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225775.920139:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1369 00000100:00100000:1.0:1761225775.920141:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720613824:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761225775.920143:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761225775.920150:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225775.920163:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225775.920181:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720614016/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225775.920204:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225775.920208:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225775.920235:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720613888:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225775.920247:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720613952:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225775.920334:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720614016:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225775.928244:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720613824:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225775.930483:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720613952:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225775.930642:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720613888:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225775.957317:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225775.957331:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225775.957342:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720614016:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225775.983993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225776.496173:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225777.008003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225777.521415:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225778.032261:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225778.544963:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225779.056027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225779.568214:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225780.079991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225780.591999:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225781.040175:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225781.040183:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225781.040187:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225781.040198:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1374 00000100:00080000:1.0:1761225781.040202:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761225781.040209:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225781.040220:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225781.040236:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720614272/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225781.040296:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225781.040300:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225781.040328:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225781.040427:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761225781.040434:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720614080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225781.040459:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225781.041614:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720614272:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225781.045421:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225781.045615:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225781.045633:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720614080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225781.048664:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225781.048674:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225781.048682:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720614272:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225781.104490:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225781.616282:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225782.127997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225782.641735:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225783.151967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225783.664172:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225784.176133:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225784.688056:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225785.199973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225785.712111:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225786.160119:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225786.160126:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225786.160130:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225786.160141:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1379 00000100:00080000:2.0:1761225786.160145:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761225786.160150:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225786.160160:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225786.160174:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720614528/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225786.160227:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225786.160231:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225786.160258:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720614400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225786.160357:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761225786.160364:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720614336:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225786.160387:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720614464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225786.160540:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720614528:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225786.166147:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720614400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225786.174500:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720614336:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225786.186497:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720614464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225786.190822:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225786.190833:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225786.190842:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720614528:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225786.223987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225786.736001:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225787.247991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225787.760813:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225788.271994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225788.787268:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225789.296300:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225789.808091:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225790.320002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225790.832121:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225791.280614:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225791.280622:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225791.280625:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225791.280638:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1384 00000100:00080000:1.0:1761225791.280641:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761225791.280664:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225791.280674:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225791.280689:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720614784/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761225791.280764:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720614784:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225791.280900:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720614592:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225791.280933:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225791.280957:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225791.280962:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614656:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761225791.281346:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225791.281350:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761225791.286388:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225791.286401:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225791.286410:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720614784:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225791.289956:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614656:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225791.289974:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720614592:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225791.289984:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225791.343971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225791.856818:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225792.367976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225792.880296:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225793.391951:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225793.907847:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225794.417304:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225794.928633:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225795.440945:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225795.952246:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225796.401199:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225796.401209:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225796.401213:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225796.401224:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1389 00000100:00080000:1.0:1761225796.401226:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761225796.401232:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225796.401243:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225796.401260:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720615040/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225796.401286:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225796.401290:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225796.401335:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225796.401426:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761225796.401432:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720614848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225796.401456:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225796.401737:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720615040:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225796.407040:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720614848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225796.407065:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225796.407101:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720614912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225796.407385:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225796.407390:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225796.407395:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720615040:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761225796.466677:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225796.976177:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225797.487990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225798.000956:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225798.512010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225799.025357:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225799.540501:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225800.048846:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225800.560047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225801.072557:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225801.520058:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225801.520079:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225801.520083:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225801.520093:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1394 00000100:00080000:1.0:1761225801.520097:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761225801.520103:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225801.520113:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225801.520130:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aa6e6fe7 x1846777720615296/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225801.520158:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225801.520161:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225801.520203:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720615168:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225801.520296:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761225801.520302:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720615104:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225801.520328:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720615232:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225801.520889:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720615296:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225801.524815:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720615104:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225801.525105:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720615232:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:02020000:3.0:1761225801.525410:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:3.0:1761225801.534256:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225801.534267:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225801.534279:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720615296:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225801.536578:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720615168:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225801.584362:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225802.096137:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225802.608061:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225803.120157:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225803.637216:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225804.144107:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225804.656000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225805.168231:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225805.679986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225806.192667:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225806.640139:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225806.640147:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225806.640150:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225806.640160:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1399 00000100:00080000:0.0:1761225806.640164:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761225806.640170:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225806.640180:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225806.640195:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720615552/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225806.640224:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225806.640228:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225806.640255:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720615424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225806.640342:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761225806.640348:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720615360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225806.640391:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720615488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225806.640483:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720615552:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761225806.644202:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225806.644213:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225806.644222:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720615552:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761225806.647251:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720615424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225806.647284:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720615360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225806.647293:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720615488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225806.703987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225807.216094:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225807.728567:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225808.241541:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225808.751933:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225809.264056:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225809.776581:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225810.288359:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225810.800016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225811.313328:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225811.760076:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225811.760086:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225811.760091:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225811.760115:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1404 00000100:00080000:0.0:1761225811.760119:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761225811.760125:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225811.760137:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225811.760155:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000024af3661 x1846777720615808/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225811.760170:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225811.760172:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225811.760196:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720615616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225811.760282:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720615744:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225811.760318:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720615680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761225811.760427:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720615872:192.168.202.114@tcp:17:kworker/1:1.0 00000100:00100000:0.0:1761225811.760447:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720615936:192.168.202.114@tcp:17:kworker/1:1.0 00000100:00100000:3.0:1761225811.760938:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720615808:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225811.763037:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720615872:192.168.202.114@tcp:17:kworker/1:1.0 00000100:00100000:1.0:1761225811.763052:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720615680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761225811.763191:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720615744:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761225811.763365:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225811.763370:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225811.763375:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720615808:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761225811.765306:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720615936:192.168.202.114@tcp:17:kworker/1:1.0 00000100:00100000:1.0:1761225811.765325:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720615616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761225811.824913:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225812.336059:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225812.847980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225813.361904:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225813.872258:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225814.384717:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225814.896395:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225815.409274:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225815.920031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225816.434351:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225816.880141:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225816.880149:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225816.880153:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225816.880167:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1410 00000100:00080000:1.0:1761225816.880173:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1761225816.880181:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720616000:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761225816.880184:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225816.880198:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225816.880223:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d0d44bb x1846777720616192/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225816.880245:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225816.880248:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225816.880292:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720616064:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225816.880321:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720616128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225816.880396:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720616192:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761225816.887666:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720616064:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225816.888036:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720616000:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761225816.892508:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225816.892521:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225816.892530:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720616192:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761225816.892624:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720616128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225816.943999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225817.456174:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225817.967964:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225818.480138:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225818.994501:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225819.504407:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225820.017470:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225820.529678:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225821.040055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225821.552156:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225822.000126:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225822.000135:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225822.000139:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225822.000152:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1415 00000100:00080000:1.0:1761225822.000156:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761225822.000163:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225822.000175:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225822.000196:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720616448/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761225822.000284:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720616448:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761225822.000520:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720616256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225822.000554:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720616384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225822.000577:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225822.000581:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720616320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761225822.004582:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225822.004600:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225822.007046:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720616320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225822.007065:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720616256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225822.007074:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720616384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761225822.008164:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225822.008173:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225822.008179:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720616448:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225822.067550:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225822.576784:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225823.088025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225823.602217:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225824.113113:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225824.624065:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225825.136007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225825.649706:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225826.159983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225826.672096:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225827.122074:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720616512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761225827.125533:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225827.125541:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225827.125545:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225827.125556:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1420 00000100:00080000:1.0:1761225827.125560:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761225827.125566:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225827.125578:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225827.125605:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ef613dc x1846777720616704/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225827.125647:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225827.125651:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225827.125687:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720616640:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225827.125689:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720616704:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761225827.125771:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720616576:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225827.126421:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720616512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225827.133506:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720616576:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761225827.133610:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225827.133615:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225827.133619:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720616704:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761225827.133702:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720616640:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225827.183984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225827.695988:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225828.208399:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225828.720058:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225829.235091:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225829.744081:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225830.257434:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225830.768185:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225831.279979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225831.792184:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225832.241365:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225832.241373:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225832.241378:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225832.241389:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1425 00000100:00080000:3.0:1761225832.241392:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761225832.241398:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225832.241408:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225832.241424:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720616960/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225832.241451:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225832.241455:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225832.241481:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720616832:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225832.241600:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761225832.241607:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720616768:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225832.241631:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720616896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225832.241957:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720616960:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225832.247159:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720616832:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225832.252708:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720616768:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761225832.254381:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225832.254390:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225832.254398:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720616960:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225832.254705:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720616896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225832.304029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225832.816094:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225833.331237:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225833.840155:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225834.351982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225834.864266:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225835.377469:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225835.891222:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225836.399991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225836.913536:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225837.360059:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225837.360068:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225837.360072:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225837.360084:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1430 00000100:00080000:3.0:1761225837.360088:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761225837.360094:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225837.360105:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225837.360119:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d56d40dd x1846777720617216/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225837.360145:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225837.360149:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225837.360196:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720617088:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225837.360298:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761225837.360304:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720617024:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225837.360327:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720617152:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225837.360419:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720617216:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:0.0:1761225837.365559:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225837.365574:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225837.365585:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720617216:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225837.366353:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720617024:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225837.366373:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720617152:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225837.366395:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720617088:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225837.424956:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225837.940451:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225838.447973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225838.960120:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225839.472688:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225839.984094:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225840.496171:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225841.008174:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225841.519975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225842.032074:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225842.482208:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225842.482216:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225842.482219:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225842.482229:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1435 00000100:00080000:1.0:1761225842.482232:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761225842.482238:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225842.482247:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225842.482263:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ef613dc x1846777720617472/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761225842.482329:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720617472:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761225842.482464:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720617280:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225842.482520:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720617408:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225842.482546:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225842.482551:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720617344:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761225842.485101:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225842.485113:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225842.486926:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720617344:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225842.486943:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720617408:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225842.487275:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720617280:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761225842.491194:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225842.491207:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225842.491217:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720617472:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225842.547126:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225843.060331:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225843.567977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225844.080922:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225844.591988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225845.104283:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225845.616097:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225846.128054:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225846.640176:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225847.157879:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225847.600128:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225847.600156:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225847.600161:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225847.600171:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1440 00000100:00080000:1.0:1761225847.600175:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761225847.600181:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225847.600193:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225847.600209:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a2fc13a x1846777720617728/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225847.600230:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225847.600233:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225847.600264:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720617600:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225847.600351:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761225847.600358:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720617536:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225847.600379:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720617664:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225847.601259:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720617728:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761225847.613826:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720617536:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225847.613876:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720617664:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225847.613892:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720617600:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761225847.614767:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225847.614774:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225847.614781:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720617728:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225847.664050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225848.176441:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225848.687986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225849.202085:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225849.711985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225850.224145:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225850.735987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225851.248082:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225851.759996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225852.272058:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225852.720219:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225852.720227:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225852.720231:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225852.720243:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1445 00000100:00080000:2.0:1761225852.720247:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761225852.720253:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225852.720264:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225852.720280:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008451bff4 x1846777720617984/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225852.720353:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720617984:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225852.720505:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720617792:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225852.720546:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720617920:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225852.720581:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225852.720586:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720617856:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225852.725147:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225852.725157:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761225852.726272:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225852.726280:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225852.726289:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720617984:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225852.726414:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720617792:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225852.731690:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720617856:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225852.731711:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720617920:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225852.785097:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225853.296251:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225853.807972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225854.320764:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225854.835017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225855.344180:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225855.856077:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225856.368405:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225856.880332:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225857.392049:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225857.840198:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225857.840207:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225857.840210:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225857.840221:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1450 00000100:00080000:0.0:1761225857.840225:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761225857.840232:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225857.840244:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225857.840252:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720618048:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225857.840263:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d88fcfe6 x1846777720618240/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225857.840290:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225857.840294:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225857.840317:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720618112:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225857.840345:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720618176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225857.840451:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720618240:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761225857.843469:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720618176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225857.844278:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720618112:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225857.845836:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720618048:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761225857.849254:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225857.849283:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225857.849293:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720618240:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225857.904403:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225858.418849:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225858.927989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225859.440743:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225859.951953:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225860.464288:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225860.975974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225861.487991:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225861.999989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225862.515054:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225862.960597:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225862.960606:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225862.960610:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225862.960623:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1456 00000100:00080000:2.0:1761225862.960628:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761225862.960636:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225862.960651:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225862.960670:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000790fc921 x1846777720618496/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225862.960700:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225862.960704:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225862.960815:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720618304:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225862.961036:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720618432:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225862.961072:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225862.961077:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720618368:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225862.961157:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720618496:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225862.967366:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720618368:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225862.967390:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720618304:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225862.967401:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720618432:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225862.967511:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225862.967527:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225862.967532:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720618496:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225863.024188:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225863.536829:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225864.052561:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225864.561508:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225865.072026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225865.585983:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225866.097845:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225866.611183:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225867.119945:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225867.633181:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225868.080573:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720618560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225868.082536:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761225868.082553:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720618624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225868.082873:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225868.082880:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225868.082884:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225868.082891:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1461 00000100:00080000:1.0:1761225868.082895:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761225868.082900:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225868.082911:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225868.082927:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d0d44bb x1846777720618752/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761225868.083005:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720618752:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225868.083629:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720618688:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225868.083773:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225868.083778:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761225868.089073:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225868.089085:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225868.089110:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720618752:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225868.089257:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720618560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225868.091487:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720618688:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225868.091506:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720618624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225868.143984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225868.656910:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225869.167999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225869.682160:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225870.191982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225870.704341:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225871.216358:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225871.728117:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225872.240426:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225872.755041:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225873.200509:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225873.200517:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225873.200521:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225873.200531:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1466 00000100:00080000:3.0:1761225873.200535:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761225873.200542:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225873.200553:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225873.200569:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720618816:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225873.200571:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abb9f4e4 x1846777720619008/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225873.200603:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225873.200607:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225873.200631:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720618880:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225873.200683:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720618944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225873.203576:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720619008:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761225873.208025:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720618816:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225873.208053:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720618944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225873.208265:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720618880:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761225873.208411:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225873.208417:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225873.208422:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720619008:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225873.264263:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225873.776061:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225874.287952:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225874.800261:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225875.312015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225875.824167:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225876.335976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225876.848487:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225877.359986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225877.872260:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225878.321414:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720619072:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225878.321514:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225878.321521:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720619136:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225878.321578:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225878.321600:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225878.321603:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225878.321608:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1471 00000100:00080000:0.0:1761225878.321611:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761225878.321616:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:0.0:1761225878.321638:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720619200:192.168.202.114@tcp:400:kworker/u8:1.0 00000008:00100000:1.0:1761225878.321662:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225878.321683:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abc685ce x1846777720619264/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225878.321705:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225878.321709:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225878.322316:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720619264:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225878.328072:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720619072:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225878.328749:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720619200:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225878.328762:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720619136:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761225878.330712:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761225878.330721:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761225878.330729:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720619264:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225878.384687:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225878.897377:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225879.407916:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225879.922691:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225880.435597:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225880.945046:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225881.456056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225881.972688:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225882.480031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225882.992197:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225883.440030:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225883.440038:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225883.440042:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761225883.440049:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720619328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761225883.440053:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1476 00000100:00080000:2.0:1761225883.440059:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761225883.440067:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225883.440109:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225883.440127:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006bca6691 x1846777720619520/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225883.440159:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225883.440162:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225883.440181:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720619456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225883.440185:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720619392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225883.440217:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720619520:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761225883.444296:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720619328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761225883.446497:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225883.446508:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225883.446516:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720619520:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761225883.446565:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720619456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225883.447274:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720619392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225883.504565:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225884.017640:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225884.528002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225885.040129:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225885.552957:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225886.065595:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225886.575926:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225887.089750:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225887.600029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225888.113462:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225888.563132:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225888.563140:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225888.563143:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225888.563153:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1481 00000100:00080000:3.0:1761225888.563156:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761225888.563162:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225888.563171:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225888.563183:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002b7677b0 x1846777720619776/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225888.563228:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225888.563232:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225888.563319:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720619776:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225888.563492:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720619584:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225888.563527:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720619712:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225888.563551:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761225888.563556:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720619648:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225888.566185:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720619584:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225888.567027:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225888.567034:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225888.567041:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720619776:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225888.567123:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720619648:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225888.570238:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720619712:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225888.623989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225889.136476:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225889.650748:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225890.160632:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225890.672010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225891.187087:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225891.696005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225892.208088:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225892.720048:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225893.232082:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225893.681122:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225893.681132:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225893.681136:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225893.681147:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1486 00000100:00100000:1.0:1761225893.681148:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720619840:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225893.681151:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225893.681157:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225893.681167:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225893.681181:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720620032/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225893.681207:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225893.681211:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225893.681228:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720619904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225893.681232:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720619968:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225893.681748:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720620032:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225893.693547:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720619968:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225893.694580:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720619840:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225893.694804:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720619904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225893.694809:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225893.694814:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225893.694818:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720620032:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225893.743991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225894.256005:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225894.768012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225895.280072:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225895.792070:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225896.304058:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225896.815990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225897.329041:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225897.840252:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225898.352502:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225898.800146:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720620096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225898.802902:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761225898.802921:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720620160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761225898.805128:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225898.805135:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225898.805139:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225898.805149:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1491 00000100:00080000:3.0:1761225898.805153:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225898.805159:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225898.805169:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225898.805185:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000727a12f1 x1846777720620288/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225898.805213:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225898.805216:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225898.805241:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225898.805258:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225898.805475:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720620288:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761225898.813364:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225898.813377:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225898.813403:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720620288:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225898.813948:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225898.813977:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720620160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225898.813986:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720620096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225898.869177:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225899.376265:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225899.888012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225900.403836:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225900.912686:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225901.424639:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225901.935991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225902.448690:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225902.959975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225903.472140:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225903.922433:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225903.922442:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225903.922446:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225903.922457:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1497 00000100:00080000:2.0:1761225903.922460:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225903.922467:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225903.922477:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225903.922492:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fab9894 x1846777720620544/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225903.922517:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225903.922521:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225903.922592:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720620544:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225903.922779:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720620352:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225903.922816:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225903.922840:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225903.922845:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620416:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225903.924680:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620416:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225903.924698:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225903.925085:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720620352:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761225903.925933:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225903.925940:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225903.925946:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720620544:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225903.983972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225904.497215:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225905.008330:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225905.520043:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225906.037751:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225906.544052:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225907.055976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225907.568562:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225908.081999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225908.596916:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225909.040121:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225909.040130:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225909.040149:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225909.040160:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1502 00000100:00080000:3.0:1761225909.040163:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225909.040168:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225909.040179:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225909.040194:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720620800/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225909.040218:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225909.040221:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225909.040248:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620672:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225909.040349:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761225909.040355:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720620608:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225909.040381:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620736:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225909.043209:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720620800:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225909.048599:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620736:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225909.048621:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720620672:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225909.053138:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225909.053150:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225909.053160:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720620800:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225909.056128:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720620608:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225909.107676:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225909.616074:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225910.127987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225910.640195:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225911.153612:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225911.664311:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225912.176717:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225912.688027:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225913.199995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225913.712113:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225914.160032:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225914.160059:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225914.160063:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225914.160074:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1507 00000100:00080000:3.0:1761225914.160078:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225914.160084:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225914.160094:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225914.160112:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720621056/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225914.160141:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225914.160145:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225914.160172:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720620928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225914.160192:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720621056:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225914.160372:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761225914.160380:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720620864:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225914.160411:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720620992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761225914.164794:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225914.164807:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225914.164816:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720621056:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225914.165107:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720620864:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225914.165120:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720620992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225914.165129:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720620928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225914.224013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225914.736038:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225915.248649:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225915.763706:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225916.271998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225916.786106:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225917.296985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225917.807983:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225918.319976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225918.833696:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225919.280094:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225919.280101:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225919.280106:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225919.280116:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1512 00000100:00080000:0.0:1761225919.280120:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225919.280126:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225919.280136:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225919.280150:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018849446 x1846777720621312/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225919.280170:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225919.280173:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225919.280195:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720621120:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225919.280212:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720621312:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225919.280446:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720621248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225919.280508:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225919.280517:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720621184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225919.283687:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720621184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225919.283705:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720621120:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225919.283714:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720621248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225919.283824:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225919.283830:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225919.283835:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720621312:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225919.343985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225919.856167:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225920.369364:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225920.880017:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225921.391979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225921.904092:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225922.415986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225922.928516:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225923.440721:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225923.952004:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225924.401425:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225924.401434:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225924.401439:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225924.401451:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1517 00000100:00080000:1.0:1761225924.401455:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225924.401463:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225924.401480:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225924.401498:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aa6e6fe7 x1846777720621568/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225924.401548:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225924.401552:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225924.401708:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720621568:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225924.402463:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720621376:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225924.402557:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720621504:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225924.402605:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761225924.402612:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720621440:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225924.412034:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720621440:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225924.412052:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720621376:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225924.412062:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720621504:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225924.413056:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225924.413065:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225924.413072:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720621568:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225924.464937:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225924.977186:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225925.489145:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225926.000272:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225926.512770:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225927.024082:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225927.537053:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225928.048878:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225928.560109:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225929.072308:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225929.522618:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225929.522627:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225929.522630:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225929.522642:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1522 00000100:00080000:0.0:1761225929.522658:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225929.522665:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225929.522678:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225929.522697:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720621824/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225929.522722:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225929.522725:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225929.522755:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720621696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225929.522915:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761225929.522923:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720621632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225929.522949:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720621760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225929.525365:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720621824:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761225929.532125:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225929.532138:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225929.532150:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720621824:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761225929.532254:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720621632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225929.532270:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720621760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225929.532282:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720621696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761225929.585149:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225930.100605:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225930.613215:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225931.120298:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225931.631964:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225932.144429:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225932.655994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761225933.168016:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225933.680038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225934.192318:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225934.641426:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720621888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225934.641821:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761225934.641822:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720621952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225934.641827:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225934.641830:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225934.641835:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1527 00000100:00080000:2.0:1761225934.641838:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225934.641895:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225934.641906:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225934.641921:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720622080/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761225934.641935:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225934.641938:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225934.641972:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720622016:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225934.642098:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720622080:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225934.650266:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720622016:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225934.650369:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720621952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225934.653437:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225934.653449:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225934.653460:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720622080:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225934.653549:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720621888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225934.706530:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225935.220457:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225935.727984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225936.240179:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225936.751920:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225937.264076:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225937.775972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225938.295318:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225938.799982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225939.312087:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225939.761131:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225939.761139:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225939.761143:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225939.761154:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1532 00000100:00080000:0.0:1761225939.761158:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225939.761164:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225939.761173:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225939.761189:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b0db11ae x1846777720622336/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225939.761214:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225939.761217:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225939.761240:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720622208:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225939.761337:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761225939.761343:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720622144:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225939.761365:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720622272:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225939.762110:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720622336:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225939.770360:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720622208:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225939.771333:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225939.771340:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225939.771347:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720622336:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761225939.772751:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720622144:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225939.772771:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720622272:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225939.823985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225940.336623:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225940.847992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225941.360769:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225941.871981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225942.385147:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225942.895992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225943.408150:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225943.921445:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225944.433780:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761225944.880259:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225944.880268:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761225944.880290:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761225944.880302:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1538 00000100:00080000:3.0:1761225944.880305:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761225944.880311:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761225944.880322:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761225944.880339:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000727a12f1 x1846777720622592/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761225944.880364:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761225944.880367:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225944.880426:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720622592:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225944.880924:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720622464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225944.880975:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761225944.880984:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720622400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225944.881006:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720622528:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761225944.887354:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761225944.887367:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761225944.887378:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720622592:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225944.887473:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720622528:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225944.888259:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720622400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225944.888276:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720622464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225944.944176:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225945.456761:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225945.968018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225946.480095:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225946.992076:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225947.504217:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225948.015991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225948.528003:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225949.039982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225949.552056:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225950.000801:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720622656:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225950.001164:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225950.001173:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720622720:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225950.001401:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225950.001407:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225950.001411:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225950.001417:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1543 00000100:00080000:2.0:1761225950.001420:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225950.001426:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225950.001437:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225950.001451:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007221726f x1846777720622848/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761225950.001506:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720622848:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761225950.002517:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225950.002524:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225950.002595:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761225950.002818:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720622784:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225950.003485:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720622656:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225950.009945:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720622784:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225950.010129:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225950.010136:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225950.010141:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720622848:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225950.010226:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720622720:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225950.064724:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225950.576419:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225951.088006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225951.600006:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225952.111987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225952.624166:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225953.135981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225953.648499:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225954.159995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225954.674700:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225955.121923:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225955.121934:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225955.121938:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225955.121964:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1548 00000100:00080000:0.0:1761225955.121967:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225955.121973:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225955.121982:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225955.122000:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720623104/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225955.122025:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225955.122029:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225955.122062:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720623104:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761225955.122063:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720622912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225955.124171:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720622976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225955.125029:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720623040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225955.132293:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720622976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225955.132293:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720622912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:02020000:2.0:1761225955.132512:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00100000:1.0:1761225955.135071:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720623040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761225955.136508:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225955.136517:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225955.136526:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720623104:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761225955.184445:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225955.696004:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225956.207974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225956.723058:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225957.231969:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225957.744069:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225958.256144:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225958.771823:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225959.280234:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225959.792551:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225960.240070:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225960.240078:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225960.240083:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225960.240094:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1553 00000100:00080000:0.0:1761225960.240097:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225960.240103:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225960.240114:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225960.240129:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000525c9db8 x1846777720623360/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225960.240151:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225960.240173:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225960.240184:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720623360:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761225960.240275:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720623232:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225960.240314:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761225960.240319:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720623168:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225960.240335:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720623296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761225960.244435:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225960.244458:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225960.244470:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720623360:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761225960.244907:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720623168:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225960.247567:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720623296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225960.247610:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720623232:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225960.303973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225960.820898:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225961.328007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225961.840353:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225962.352157:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225962.864389:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225963.376023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225963.888082:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225964.400042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225964.912604:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761225965.360525:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720623424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225965.360963:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225965.360971:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720623488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761225965.362791:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225965.362799:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225965.362803:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225965.362813:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1558 00000100:00080000:1.0:1761225965.362817:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225965.362823:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225965.362833:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225965.362848:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017ee9697 x1846777720623616/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225965.362884:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225965.362888:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225965.362906:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761225965.362920:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720623552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225965.363210:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720623616:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761225965.370115:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720623552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225965.370238:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720623488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225965.370249:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720623424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761225965.370593:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225965.370598:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225965.370603:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720623616:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761225965.423979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225965.939069:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225966.447992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225966.963704:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225967.473293:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225967.984333:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225968.496161:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225969.008420:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225969.519980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225970.032256:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225970.480043:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225970.480052:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225970.480055:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225970.480072:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1563 00000100:00080000:2.0:1761225970.480075:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225970.480082:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225970.480092:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225970.480109:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000003513afe x1846777720623872/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225970.480176:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720623872:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225970.481453:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720623680:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225970.481592:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225970.481597:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225970.481633:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720623744:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225970.491658:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720623808:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225970.497417:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225970.497430:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225970.497442:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720623872:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761225970.503769:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720623744:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225970.503969:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720623680:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225970.503981:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720623808:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225970.543952:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225971.060010:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225971.567984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225972.080697:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225972.592555:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225973.111239:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225973.618699:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225974.128095:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225974.640070:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225975.152094:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225975.600802:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225975.600811:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225975.600815:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225975.600826:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1568 00000100:00080000:1.0:1761225975.600830:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225975.600837:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225975.600847:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225975.600904:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017ee9697 x1846777720624128/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225975.600918:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225975.600921:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761225975.600948:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720623936:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225975.601032:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720624064:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761225975.601062:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761225975.601068:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720624000:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225975.601417:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720624128:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761225975.607067:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225975.607078:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225975.607087:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720624128:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225975.607216:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720624000:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225975.607229:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720624064:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225975.608331:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720623936:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761225975.663974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225976.179035:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225976.688001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761225977.209189:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225977.711985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225978.226155:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225978.735976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225979.250442:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225979.760070:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225980.272046:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225980.720087:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225980.720095:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225980.720099:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225980.720109:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1573 00000100:00080000:1.0:1761225980.720113:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225980.720119:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225980.720148:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225980.720164:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720624384/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225980.720194:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225980.720198:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225980.720290:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720624192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225980.720393:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720624320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225980.720434:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761225980.720440:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720624256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225980.720744:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720624384:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761225980.729365:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225980.729377:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225980.729397:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720624384:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761225980.729921:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720624256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225980.729934:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720624192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225980.729944:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720624320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225980.784006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225981.298899:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225981.808228:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225982.322028:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225982.831952:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225983.345833:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761225983.858068:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225984.370927:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225984.886211:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225985.392014:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761225985.840385:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225985.840392:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761225985.840411:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761225985.840420:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1578 00000100:00080000:2.0:1761225985.840423:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761225985.840438:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761225985.840448:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761225985.840463:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001379e0d8 x1846777720624640/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761225985.840521:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720624640:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225985.841346:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720624448:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225985.842832:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720624576:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225985.843052:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761225985.843057:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761225985.843230:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761225985.843238:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720624512:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761225985.846280:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761225985.846292:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761225985.846301:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720624640:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225985.846658:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720624512:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225985.846672:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720624576:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225985.848140:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720624448:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761225985.903987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225986.416079:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225986.928000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225987.440065:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225987.952000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225988.464147:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225988.976113:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225989.488704:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225989.999975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225990.512281:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761225990.961059:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225990.961069:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761225990.961073:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761225990.961084:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1584 00000100:00080000:1.0:1761225990.961088:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761225990.961095:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761225990.961104:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761225990.961119:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057d87282 x1846777720624896/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761225990.961151:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761225990.961155:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761225990.961419:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720624896:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761225990.961887:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720624704:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225990.961925:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720624832:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225990.961950:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761225990.961956:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720624768:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225990.967179:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720624768:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225990.967196:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720624704:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761225990.967205:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720624832:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761225990.967426:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225990.967431:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225990.967436:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720624896:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761225991.024038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225991.537307:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225992.048581:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225992.560112:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225993.073981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225993.584719:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225994.095977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225994.608795:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225995.119982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225995.633432:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761225996.080104:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225996.080112:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761225996.080116:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761225996.080127:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1589 00000100:00080000:0.0:1761225996.080130:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761225996.080136:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761225996.080146:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761225996.080160:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000525c9db8 x1846777720625152/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761225996.080183:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761225996.080186:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761225996.080210:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625024:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225996.080293:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761225996.080299:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720624960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761225996.080321:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761225996.081473:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720625152:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761225996.089216:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720624960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225996.089237:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761225996.089246:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625024:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761225996.089344:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761225996.089350:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761225996.089354:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720625152:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761225996.144458:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225996.656311:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225997.168139:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761225997.680329:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225998.192085:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225998.706004:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761225999.216016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761225999.728204:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226000.239976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226000.752126:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226001.200082:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226001.200091:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226001.200095:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226001.200105:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1594 00000100:00080000:0.0:1761226001.200109:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226001.200133:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226001.200149:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226001.200164:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720625216:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226001.200170:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d991e6ad x1846777720625408/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226001.200201:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226001.200204:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226001.200227:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720625280:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226001.200265:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625344:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226001.200486:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720625408:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226001.204083:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720625280:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226001.204170:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720625216:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226001.204182:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625344:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226001.208675:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226001.208686:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226001.208694:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720625408:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226001.263984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226001.776204:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226002.287972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226002.802321:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226003.311983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226003.824055:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226004.339730:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226004.849087:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226005.362281:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226005.872081:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226006.320163:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226006.320170:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226006.320174:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226006.320184:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1599 00000100:00080000:1.0:1761226006.320187:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226006.320193:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226006.320203:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226006.320216:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720625664/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226006.320243:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226006.320247:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226006.320273:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625536:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226006.320357:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761226006.320364:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720625472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226006.320390:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625600:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226006.320511:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720625664:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226006.329662:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720625472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226006.329687:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625600:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226006.329698:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720625536:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226006.330495:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226006.330501:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226006.330507:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720625664:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226006.387942:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226006.896079:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226007.407996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226007.920902:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226008.431975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226008.944091:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226009.455993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226009.969580:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226010.480154:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226010.991994:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226011.440980:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226011.440990:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226011.440994:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226011.441005:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1604 00000100:00080000:0.0:1761226011.441008:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226011.441016:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226011.441027:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226011.441046:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720625920/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226011.441071:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226011.441075:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226011.441103:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720625792:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226011.441194:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761226011.441201:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720625728:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226011.441242:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720625856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226011.441754:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720625920:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761226011.447650:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226011.447665:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226011.447675:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720625920:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226011.450751:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720625728:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226011.450790:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720625856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226011.450802:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720625792:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226011.504025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226012.016285:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226012.527957:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226013.040238:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226013.552257:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226014.064147:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226014.576009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226015.088050:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226015.599969:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226016.112241:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226016.560159:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226016.560168:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226016.560172:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226016.560183:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1609 00000100:00100000:1.0:1761226016.560184:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720625984:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226016.560187:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226016.560194:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226016.560203:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226016.560220:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720626176/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226016.560243:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226016.560246:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226016.560274:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720626048:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226016.560298:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720626112:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226016.560520:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720626176:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761226016.567589:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720626048:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226016.567590:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720625984:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226016.567614:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720626112:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226016.571722:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226016.571735:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226016.571744:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720626176:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226016.624266:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226017.136595:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226017.648552:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226018.160247:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226018.671983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226019.184126:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226019.696560:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226020.209808:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226020.720354:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226021.232209:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226021.680072:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226021.680081:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226021.680084:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226021.680095:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1614 00000100:00080000:0.0:1761226021.680099:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226021.680105:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226021.680135:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226021.680191:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720626432/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226021.680216:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226021.680219:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226021.680249:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720626304:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226021.680352:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761226021.680358:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720626240:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226021.680384:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720626368:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226021.680609:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720626432:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761226021.696698:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720626240:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226021.696718:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720626368:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226021.696727:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720626304:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226021.699920:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226021.699931:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226021.699941:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720626432:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226021.745328:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226022.256186:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226022.768881:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226023.282121:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226023.792216:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226024.304490:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226024.815962:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226025.328261:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226025.839996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226026.352105:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226026.800337:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226026.800345:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226026.800349:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226026.800359:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1619 00000100:00080000:2.0:1761226026.800379:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226026.800385:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226026.800396:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226026.800410:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046bdcfa1 x1846777720626688/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226026.800426:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226026.800429:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226026.800794:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720626688:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226026.805010:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720626496:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226026.805221:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720626560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226026.809899:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720626624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761226026.813184:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226026.813197:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226026.813208:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720626688:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226026.822490:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720626560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226026.823349:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720626496:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226026.823367:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720626624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226026.865012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226027.376370:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226027.889008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226028.400270:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226028.911975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226029.424351:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226029.936017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226030.448346:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226030.963020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226031.476188:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226031.920067:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226031.920077:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226031.920080:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226031.920091:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1625 00000100:00080000:2.0:1761226031.920094:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226031.920100:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226031.920111:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226031.920132:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000790fc921 x1846777720626944/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226031.920152:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761226031.920157:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720626752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226031.920159:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226031.920212:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720626816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226031.920287:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720626880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226031.920572:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720626944:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226031.926179:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720626816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226031.926721:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720626752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226031.926745:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720626880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226031.927266:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226031.927272:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226031.927277:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720626944:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226031.984982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226032.497817:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226033.009136:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226033.520827:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226034.032795:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226034.545220:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226035.056064:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226035.569101:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226036.079980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226036.592510:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226037.040420:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226037.040453:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226037.040458:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226037.040468:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1630 00000100:00080000:2.0:1761226037.040472:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226037.040478:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226037.040489:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226037.040506:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000064ce9b52 x1846777720627200/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226037.040528:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226037.040531:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226037.040565:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720627200:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226037.041063:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720627008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226037.041098:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720627136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226037.041117:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226037.041121:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720627072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226037.046745:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720627072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226037.046763:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720627008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226037.046772:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720627136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226037.047192:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226037.047197:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226037.047207:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720627200:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226037.103985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226037.617233:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226038.127954:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226038.640063:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226039.152516:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226039.664205:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226040.176073:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226040.688260:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226041.199994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226041.712324:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226042.161542:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720627264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226042.163606:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226042.163620:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720627328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226042.165373:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226042.165381:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226042.165385:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226042.165394:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1635 00000100:00080000:2.0:1761226042.165397:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226042.165419:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226042.165430:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226042.165446:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a37cdda5 x1846777720627456/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226042.165471:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226042.165475:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226042.165498:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226042.165512:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720627392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226042.165731:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720627456:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226042.169724:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720627264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226042.172510:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226042.172521:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226042.172531:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720627456:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226042.174780:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720627328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226042.174900:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720627392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226042.225825:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226042.737062:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226043.248418:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226043.761663:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226044.272126:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226044.784938:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226045.296781:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226045.808102:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226046.320008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226046.832101:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226047.280523:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720627520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226047.280714:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226047.280720:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226047.280724:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226047.280730:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1640 00000100:00080000:1.0:1761226047.280733:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226047.280740:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226047.280753:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226047.280769:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720627712/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226047.280811:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226047.280815:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226047.280906:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720627648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226047.280936:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226047.280946:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720627584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226047.281651:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720627712:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226047.285183:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720627648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226047.297052:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720627584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226047.297075:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720627520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226047.298686:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226047.298700:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226047.298710:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720627712:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226047.347055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226047.856138:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226048.369937:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226048.880162:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226049.392025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226049.907019:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226050.415965:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226050.930658:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226051.440023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226051.952681:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226052.400133:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226052.400141:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226052.400145:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226052.400155:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1645 00000100:00080000:2.0:1761226052.400160:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226052.400167:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226052.400178:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226052.400192:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b3214ada x1846777720627968/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226052.400241:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226052.400245:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226052.400325:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720627776:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226052.400979:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720627968:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226052.401170:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720627904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226052.401485:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720627840:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226052.404403:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720627840:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226052.405165:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226052.405173:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226052.405178:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720627968:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226052.405330:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720627776:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226052.405361:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720627904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226052.464426:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226052.983912:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226053.488033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226054.000062:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226054.512265:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226055.025738:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226055.538069:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226056.049301:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226056.559939:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226057.072001:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226057.520360:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226057.520369:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226057.520372:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226057.520383:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1650 00000100:00080000:1.0:1761226057.520403:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226057.520410:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226057.520420:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226057.520437:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ef613dc x1846777720628224/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226057.520464:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226057.520468:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226057.520493:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720628096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226057.520526:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720628032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226057.520632:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720628224:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226057.520752:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226057.527491:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720628096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226057.528156:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720628032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226057.528171:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226057.528267:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226057.528272:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226057.528276:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720628224:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226057.584033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226058.096157:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226058.607972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226059.119994:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226059.632800:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226060.144020:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226060.655985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226061.168202:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226061.679998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226062.192015:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226062.640048:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226062.640056:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226062.640060:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226062.640071:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1655 00000100:00080000:1.0:1761226062.640074:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226062.640080:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226062.640090:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226062.640109:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ef613dc x1846777720628480/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226062.640135:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226062.640139:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226062.640174:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226062.640272:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761226062.640288:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720628480:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226062.640291:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720628288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226062.640326:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226062.647932:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226062.647945:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226062.647955:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720628480:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226062.649138:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226062.650504:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226062.655552:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720628288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226062.704122:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226063.217111:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226063.727980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226064.240264:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226064.751943:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226065.265961:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226065.775967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226066.288093:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226066.800002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226067.312068:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226067.760071:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226067.760081:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226067.760085:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226067.760097:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1660 00000100:00080000:1.0:1761226067.760101:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226067.760108:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226067.760120:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226067.760135:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720628736/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226067.760164:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226067.760177:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226067.760197:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720628544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226067.760197:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720628608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226067.760297:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720628736:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226067.760350:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226067.765774:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720628608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226067.765905:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226067.767606:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226067.767615:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226067.767622:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720628736:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226067.768661:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720628544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226067.823987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226068.336759:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226068.848060:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226069.362685:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226069.871995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226070.386207:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226070.895981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226071.409426:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226071.921797:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226072.435365:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226072.880093:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226072.880104:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226072.880108:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226072.880120:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1666 00000100:00080000:1.0:1761226072.880124:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226072.880130:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226072.880141:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226072.880158:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b1564d60 x1846777720628992/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226072.880201:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226072.880205:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226072.880268:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720628992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226072.880447:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226072.880491:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226072.880497:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720628800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226072.880519:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226072.886458:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226072.886479:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720628864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226072.887839:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226072.887890:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226072.887898:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720628992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226072.889636:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720628800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226072.945577:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226073.457518:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226073.967988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226074.480416:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226074.991974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226075.504199:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226076.016026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226076.528033:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226077.039995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226077.555506:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226078.000084:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226078.000092:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226078.000096:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226078.000105:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1671 00000100:00080000:2.0:1761226078.000109:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226078.000115:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226078.000125:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226078.000139:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720629248/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226078.000164:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226078.000167:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226078.000247:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720629248:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226078.000974:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720629056:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226078.002698:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720629184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226078.002871:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226078.002879:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720629120:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226078.006788:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226078.006800:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226078.006811:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720629248:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226078.008343:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720629120:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226078.008367:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720629056:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226078.008379:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720629184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226078.064641:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226078.580254:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226079.087993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226079.601577:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226080.111993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226080.624577:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226081.136036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226081.648524:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226082.159960:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226082.672567:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226083.120078:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226083.120087:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226083.120091:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226083.120102:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1676 00000100:00080000:2.0:1761226083.120106:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226083.120112:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226083.120122:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226083.120137:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a37cdda5 x1846777720629504/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226083.120163:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226083.120184:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226083.120284:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720629312:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226083.120382:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720629440:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226083.120412:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226083.120418:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720629376:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226083.121223:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720629504:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226083.129103:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720629312:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226083.130836:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720629376:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226083.130897:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720629440:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226083.139193:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226083.139206:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226083.139217:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720629504:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226083.183984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226083.696368:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226084.207997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226084.722498:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226085.232168:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226085.744332:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226086.256252:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226086.768083:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226087.279977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226087.793458:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226088.240119:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226088.240127:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226088.240131:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226088.240141:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1681 00000100:00080000:0.0:1761226088.240144:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226088.240168:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226088.240178:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226088.240194:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720629760/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226088.240218:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226088.240222:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226088.240250:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720629632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226088.240339:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761226088.240347:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720629568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226088.240384:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720629696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226088.240892:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720629760:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226088.246374:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720629568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226088.247129:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720629696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226088.247142:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720629632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226088.252281:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226088.252292:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226088.252303:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720629760:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226088.303960:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226088.816806:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226089.327978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226089.840072:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226090.353227:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226090.864398:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226091.376185:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226091.888155:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226092.399960:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226092.912152:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226093.360186:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226093.360196:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226093.360200:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226093.360210:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1686 00000100:00080000:0.0:1761226093.360213:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226093.360219:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226093.360230:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226093.360248:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720630016/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226093.360273:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226093.360277:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226093.360443:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720629824:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226093.361221:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720630016:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761226093.362072:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720629952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226093.362142:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226093.362151:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720629888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226093.365634:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720629824:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226093.371181:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720629952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226093.371371:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226093.371378:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226093.371381:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720629888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226093.371385:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720630016:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226093.423982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226093.936100:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226094.449246:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226094.960248:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226095.471982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226095.985890:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226096.495997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226097.008180:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226097.520031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226098.037415:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226098.480309:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226098.480317:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226098.480322:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761226098.480326:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630144:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226098.480334:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1691 00000100:00080000:2.0:1761226098.480342:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226098.480351:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226098.480366:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226098.480388:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720630272/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226098.480406:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226098.480410:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226098.480439:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761226098.480450:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720630080:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226098.480476:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630208:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226098.480541:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720630272:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226098.495192:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720630080:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226098.495924:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630208:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226098.495939:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630144:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226098.496137:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226098.496158:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226098.496163:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720630272:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226098.544901:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226099.056379:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226099.580852:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226100.080070:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226100.592063:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226101.104196:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226101.615974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226102.128320:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226102.641622:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226103.152640:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226103.604221:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226103.604230:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226103.604234:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226103.604245:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1696 00000100:00080000:3.0:1761226103.604248:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226103.604254:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226103.604264:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226103.604281:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003d27cd56 x1846777720630528/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226103.604367:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720630528:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226103.604595:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720630336:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226103.604632:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226103.604664:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226103.604670:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761226103.605716:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226103.605724:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761226103.613816:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226103.613829:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226103.613844:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720630528:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226103.613874:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630400:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226103.613886:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720630336:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226103.613896:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630464:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226103.666995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226104.178021:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226104.688013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226105.200123:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226105.712248:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226106.224019:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226106.736435:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226107.248014:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226107.761055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226108.272105:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226108.720106:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226108.720132:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226108.720137:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226108.720147:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1701 00000100:00080000:1.0:1761226108.720150:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226108.720157:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226108.720173:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226108.720191:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720630784/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226108.720219:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226108.720223:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226108.720252:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720630656:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226108.720267:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720630784:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226108.720362:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226108.720369:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720630592:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226108.720399:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720630720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:2.0:1761226108.727231:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00100000:1.0:1761226108.727410:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720630720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226108.727427:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720630656:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226108.729827:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720630592:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226108.737915:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226108.737925:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226108.737936:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720630784:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226108.783975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226109.300895:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226109.807993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226110.320150:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226110.831975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226111.345343:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226111.855985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226112.367997:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226112.880034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226113.391976:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226113.840477:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720630848:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226113.841832:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630912:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226113.843260:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226113.843272:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630976:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761226113.843485:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226113.843491:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226113.843494:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226113.843500:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1706 00000100:00080000:3.0:1761226113.843503:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226113.843563:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226113.843574:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226113.843592:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d56d40dd x1846777720631040/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226113.843618:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226113.843621:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226113.843960:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720631040:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226113.848253:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720630848:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226113.851462:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630976:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226113.851484:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720630912:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761226113.851592:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226113.851613:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226113.851617:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720631040:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226113.903997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226114.416844:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226114.927996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226115.440729:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226115.956286:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226116.464121:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226116.976108:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226117.488683:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226118.000012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226118.512031:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226118.960065:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226118.960073:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226118.960077:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226118.960099:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1712 00000100:00080000:3.0:1761226118.960103:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226118.960110:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226118.960132:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226118.960148:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035883416 x1846777720631296/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226118.960180:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226118.960184:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226118.961580:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720631296:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226118.962227:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720631104:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226118.962267:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720631232:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226118.962290:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226118.962295:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720631168:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226118.964725:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226118.964735:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226118.964743:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720631296:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226118.966141:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720631104:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226118.966160:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720631232:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226118.966183:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720631168:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226119.024047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226119.539584:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226120.048977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226120.560120:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226121.071983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226121.584159:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226122.095986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226122.608737:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226123.119978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226123.632481:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226124.080126:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226124.080134:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226124.080138:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226124.080149:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1717 00000100:00080000:1.0:1761226124.080152:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226124.080159:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226124.080168:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226124.080183:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720631552/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226124.080211:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226124.080214:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226124.080299:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720631360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226124.080403:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720631488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226124.080435:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226124.080441:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720631424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226124.082271:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720631552:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226124.087523:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720631488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226124.087561:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720631424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226124.087571:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720631360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226124.089259:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226124.089270:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226124.089278:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720631552:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226124.143979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226124.657541:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226125.168082:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226125.679996:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226126.196527:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226126.704215:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226127.215987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226127.730817:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226128.239980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226128.752085:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226129.204347:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226129.204355:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226129.204358:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226129.204370:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1722 00000100:00080000:3.0:1761226129.204374:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1761226129.204375:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720631616:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761226129.204384:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226129.204395:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226129.204411:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f64df33e x1846777720631808/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226129.204440:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226129.204444:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226129.204461:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720631680:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226129.204480:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720631744:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226129.204566:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720631808:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226129.211689:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720631680:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761226129.215164:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226129.215174:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226129.215184:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720631808:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226129.215394:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720631744:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226129.217217:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720631616:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226129.263978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226129.776347:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226130.287990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226130.800114:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226131.316640:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226131.824036:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226132.336007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226132.848360:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226133.360021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226133.872200:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226134.321793:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226134.321801:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226134.321805:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226134.321816:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1727 00000100:00080000:3.0:1761226134.321819:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226134.321842:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226134.321878:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226134.321896:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f9712cd6 x1846777720632064/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226134.321915:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226134.321918:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226134.322016:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720632064:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226134.322330:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720631936:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226134.322379:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761226134.322391:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720631872:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226134.322416:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720632000:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226134.324009:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720631872:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226134.324235:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720632000:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226134.326136:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720631936:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761226134.326624:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226134.326631:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226134.326636:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720632064:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226134.383973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226134.896387:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226135.407944:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226135.920091:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226136.432010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226136.947186:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226137.456022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226137.968001:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226138.480018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226138.995007:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226139.440078:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720632128:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226139.440677:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226139.440685:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720632192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226139.440962:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226139.440968:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226139.440973:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226139.440978:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1732 00000100:00080000:1.0:1761226139.440981:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226139.440987:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226139.440997:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226139.441013:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017ee9697 x1846777720632320/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226139.441039:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226139.441043:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226139.441077:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720632320:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226139.441943:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720632256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226139.444443:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720632128:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226139.453681:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226139.453693:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226139.453704:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720632320:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226139.454836:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720632256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226139.454895:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720632192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226139.504010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226140.016040:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226140.528001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226141.040070:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226141.554998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226142.064201:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226142.576282:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226143.088540:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226143.599985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226144.112581:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226144.560093:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226144.560102:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226144.560106:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226144.560118:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1737 00000100:00080000:1.0:1761226144.560122:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761226144.560129:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720632384:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226144.560130:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226144.560143:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226144.560160:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720632576/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226144.560187:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226144.560191:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226144.560223:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720632448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226144.560228:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720632512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226144.560238:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720632576:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761226144.567069:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226144.567081:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226144.567092:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720632576:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226144.567444:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720632448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226144.567476:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720632384:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226144.567486:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720632512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226144.623991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226145.136001:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226145.649974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226146.162436:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226146.672247:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226147.188892:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226147.696004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226148.208299:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226148.719987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226149.234146:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226149.681834:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226149.681843:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226149.681847:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226149.681985:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1742 00000100:00080000:3.0:1761226149.681989:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226149.681996:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226149.682025:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226149.682045:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8803538 x1846777720632832/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226149.682072:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226149.682076:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226149.682109:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720632704:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226149.682273:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720632832:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226149.682423:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720632640:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226149.686104:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720632768:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226149.693120:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720632704:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761226149.693517:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226149.693523:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226149.693529:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720632832:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226149.693540:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720632640:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226149.693558:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720632768:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226149.744009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226150.257157:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226150.768990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226151.291907:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226151.792563:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226152.305812:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226152.815982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226153.327994:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226153.842897:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226154.352318:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226154.802061:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226154.802069:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226154.802073:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226154.802084:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1747 00000100:00080000:0.0:1761226154.802087:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226154.802094:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226154.802104:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226154.802123:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720633088/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226154.802152:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226154.802155:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226154.802183:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720632960:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226154.802293:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761226154.802300:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720632896:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226154.802323:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720633024:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226154.802425:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720633088:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:2.0:1761226154.808309:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226154.808320:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226154.808331:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720633088:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761226154.808662:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720632960:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226154.808686:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720632896:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226154.808733:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720633024:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226154.863995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226155.377852:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226155.887954:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226156.400979:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226156.912783:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226157.426001:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226157.936879:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226158.450768:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226158.962940:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226159.473261:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226159.920088:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226159.920097:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226159.920101:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226159.920112:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1753 00000100:00080000:0.0:1761226159.920116:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761226159.920119:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720633152:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226159.920125:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226159.920137:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226159.920157:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720633344/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226159.920182:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226159.920186:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226159.920217:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720633280:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226159.920219:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720633216:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226159.920350:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720633344:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226159.925642:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720633216:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226159.925975:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720633280:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226159.925994:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720633152:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226159.926696:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226159.926702:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226159.926707:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720633344:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226159.984149:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226160.498220:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226161.007984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226161.520160:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226162.031976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226162.546445:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226163.055985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226163.578591:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226164.080838:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226164.598510:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226165.040280:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720633408:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226165.040576:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226165.040583:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720633472:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226165.042145:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226165.042172:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226165.042176:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226165.042185:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1758 00000100:00080000:0.0:1761226165.042189:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226165.042196:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226165.042210:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226165.042228:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b0db11ae x1846777720633600/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226165.042260:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226165.042264:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226165.042348:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226165.042367:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720633536:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226165.042844:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720633600:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226165.051294:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720633536:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226165.051491:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720633472:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226165.051504:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720633408:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226165.051591:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226165.051596:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226165.051601:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720633600:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226165.104060:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226165.616303:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226166.133050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226166.641944:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226167.152219:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226167.664346:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226168.176313:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226168.688183:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226169.199971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226169.712786:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226170.160074:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226170.160082:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226170.160086:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226170.160098:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1763 00000100:00080000:0.0:1761226170.160101:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226170.160109:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226170.160124:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226170.160143:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d991e6ad x1846777720633856/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226170.160176:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226170.160188:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226170.160215:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720633664:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226170.160287:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720633728:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226170.160317:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720633792:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226170.160768:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720633856:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761226170.164259:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720633728:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226170.164372:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720633664:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226170.164386:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720633792:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761226170.168084:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226170.168110:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226170.168120:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720633856:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226170.224159:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226170.736524:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226171.247979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226171.760205:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226172.276554:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226172.788445:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226173.295982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226173.808703:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226174.320025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226174.834752:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226175.280104:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226175.280112:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226175.280116:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226175.280127:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1768 00000100:00080000:3.0:1761226175.280131:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226175.280137:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226175.280148:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226175.280163:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a6d0bfc0 x1846777720634112/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226175.280185:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226175.280189:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226175.280249:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720634112:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:3.0:1761226175.283965:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226175.283977:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226175.283987:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720634112:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226175.283991:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720633920:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226175.284041:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720633984:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226175.284086:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720634048:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226175.292588:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720633920:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226175.297052:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720634048:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226175.298007:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720633984:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226175.343983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226175.858640:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226176.368318:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226176.880077:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226177.392253:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226177.904806:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226178.417025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226178.930067:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226179.439961:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226179.952639:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226180.400146:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720634176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226180.402744:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226180.402752:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226180.402773:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226180.402784:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1773 00000100:00080000:1.0:1761226180.402788:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226180.402794:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226180.402805:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226180.402822:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000158429d4 x1846777720634368/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226180.402867:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226180.402871:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226180.402897:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720634240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226180.403124:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720634368:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226180.404002:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720634304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226180.404059:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720634176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226180.407159:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226180.407171:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226180.407181:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720634368:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226180.407436:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720634304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226180.407560:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720634240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226180.463983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226180.976304:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226181.488031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226182.000750:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226182.511989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226183.025999:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226183.535951:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226184.048170:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226184.560065:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226185.073815:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226185.520142:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226185.520167:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226185.520171:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226185.520181:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1778 00000100:00080000:0.0:1761226185.520185:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226185.520191:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226185.520202:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226185.520218:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720634624/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226185.520243:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226185.520246:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226185.520274:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720634496:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226185.520363:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226185.520369:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720634432:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226185.520390:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720634560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226185.520687:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720634624:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761226185.524069:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720634432:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226185.527295:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720634560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226185.527315:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720634496:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226185.540404:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226185.540424:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226185.540435:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720634624:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226185.583978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226186.096156:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226186.607999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226187.120339:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226187.631979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226188.144536:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226188.660694:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226189.168049:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226189.680021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226190.193025:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226190.641150:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226190.641158:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226190.641162:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226190.641172:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1783 00000100:00080000:3.0:1761226190.641176:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226190.641182:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226190.641192:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226190.641227:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f64df33e x1846777720634880/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226190.641254:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226190.641257:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226190.641431:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720634688:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226190.641446:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720634880:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226190.641534:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720634816:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226190.641565:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761226190.641579:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720634752:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226190.645783:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720634752:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226190.645801:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720634816:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:3.0:1761226190.647420:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226190.647430:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226190.647438:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720634880:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226190.647699:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720634688:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226190.704884:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226191.216130:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226191.728410:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226192.240084:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226192.752947:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226193.264505:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226193.775984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226194.288179:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226194.800004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226195.312148:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226195.763623:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226195.763631:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226195.763635:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226195.763645:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1788 00000100:00080000:2.0:1761226195.763649:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226195.763654:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226195.763665:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226195.763679:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084e78951 x1846777720635136/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226195.763701:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226195.763704:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226195.763728:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720635008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226195.763818:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761226195.763824:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720634944:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226195.763842:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720635072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226195.763951:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720635136:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226195.769702:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720634944:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226195.770320:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226195.770328:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226195.770333:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720635136:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226195.770425:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720635072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226195.770437:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720635008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226195.832826:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226196.337563:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226196.849509:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226197.361227:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226197.872134:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226198.384585:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226198.895980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226199.408635:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226199.919990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226200.432196:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226200.880106:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226200.880115:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226200.880119:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226200.880130:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1794 00000100:00080000:1.0:1761226200.880134:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226200.880141:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226200.880172:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226200.880191:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b1564d60 x1846777720635392/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226200.880222:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226200.880226:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226200.880262:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720635264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226200.880277:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720635392:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226200.880398:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226200.880404:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720635200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226200.880428:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720635328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226200.885408:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720635200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226200.885436:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720635328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226200.885448:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720635264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226200.888900:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226200.888911:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226200.888920:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720635392:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226200.944493:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226201.456229:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226201.967985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226202.484338:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226202.992194:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226203.504253:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226204.015971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226204.531726:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226205.039981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226205.552596:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226206.000063:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226206.000072:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226206.000076:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226206.000087:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1799 00000100:00080000:1.0:1761226206.000090:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226206.000097:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226206.000109:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226206.000126:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720635648/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226206.000153:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226206.000157:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226206.000206:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720635520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226206.000213:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720635648:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226206.000325:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761226206.000332:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720635456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226206.000351:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720635584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226206.003721:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720635456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226206.003744:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720635584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226206.003755:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720635520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226206.003917:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226206.003923:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226206.003927:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720635648:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226206.063970:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226206.585009:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226207.090781:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226207.600112:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226208.116370:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226208.624049:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226209.136763:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226209.657021:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226210.159994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226210.672000:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226211.120094:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720635712:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226211.120827:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226211.120837:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720635776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226211.121174:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226211.121180:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226211.121183:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226211.121190:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1804 00000100:00080000:1.0:1761226211.121193:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226211.121198:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226211.121209:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226211.121223:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720635904/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226211.121243:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226211.121246:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226211.121299:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226211.121308:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720635840:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226211.121452:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720635904:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226211.123276:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720635712:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226211.125822:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720635840:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226211.125824:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226211.125832:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226211.125839:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720635904:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226211.129372:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720635776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226211.184248:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226211.696292:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226212.209504:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226212.720787:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226213.232182:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226213.744024:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226214.255931:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226214.769261:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226215.280002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226215.792351:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226216.241489:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226216.241498:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226216.241502:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226216.241512:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1809 00000100:00080000:2.0:1761226216.241516:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226216.241523:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226216.241535:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226216.241551:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fab9894 x1846777720636160/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226216.241580:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226216.241584:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226216.241668:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720636160:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226216.241943:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720635968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226216.242009:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720636096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226216.242062:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226216.242070:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720636032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226216.248823:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720635968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226216.249796:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720636096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226216.252401:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720636032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226216.252473:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226216.252478:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226216.252499:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720636160:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226216.307325:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226216.816387:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226217.329998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226217.840109:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226218.351973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226218.864156:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226219.375983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226219.888796:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226220.399978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226220.912039:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226221.360199:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720636224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226221.361359:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226221.361369:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226221.362557:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226221.362563:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226221.362567:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226221.362576:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1814 00000100:00080000:0.0:1761226221.362579:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226221.362585:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226221.362596:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226221.362611:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000de75897f x1846777720636416/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226221.362660:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720636416:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761226221.363694:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226221.363702:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226221.363737:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226221.372471:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1761226221.372671:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226221.372688:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226221.372909:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226221.372914:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720636416:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226221.373119:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720636224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226221.423960:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226221.942168:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226222.447987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226222.961203:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226223.472583:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226223.984165:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226224.495976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226225.008664:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226225.519995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226226.033373:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226226.480128:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226226.480136:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226226.480140:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226226.480150:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1819 00000100:00080000:1.0:1761226226.480153:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226226.480159:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226226.480169:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226226.480183:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094e95ca3 x1846777720636672/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226226.480206:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226226.480209:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226226.480235:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226226.480252:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720636672:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226226.480351:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226226.480360:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720636480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226226.480386:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226226.482352:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226226.482980:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720636480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226226.483003:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226226.483574:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226226.483580:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226226.483585:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720636672:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226226.544185:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226227.056554:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226227.568015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226228.082429:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226228.592034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226229.104156:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226229.620889:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226230.128874:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226230.640010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226231.152349:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226231.600383:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720636736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226231.600799:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226231.600806:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226231.605417:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720636800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226231.608168:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226231.608177:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226231.608182:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226231.608192:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1824 00000100:00080000:1.0:1761226231.608195:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226231.608202:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226231.608214:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226231.608232:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ef613dc x1846777720636928/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226231.608246:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226231.608249:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226231.608275:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226231.608294:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720636864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226231.608482:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720636928:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226231.617061:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720636736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226231.618317:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720636864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226231.618420:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226231.618426:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226231.618430:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720636928:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226231.667235:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226232.176152:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226232.688025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226233.200192:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226233.716053:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226234.228429:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226234.736489:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226235.249727:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226235.760019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226236.272675:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226236.720080:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226236.720090:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226236.720095:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226236.720108:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1829 00000100:00080000:1.0:1761226236.720111:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226236.720119:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226236.720130:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226236.720146:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d0d44bb x1846777720637184/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226236.720175:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226236.720178:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226236.720206:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720637056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226236.720305:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226236.720311:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720636992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226236.720334:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720637120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226236.720451:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720637184:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226236.723917:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720636992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226236.725496:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720637120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226236.725537:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720637056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226236.730791:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226236.730805:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226236.730816:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720637184:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226236.783979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226237.297453:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226237.810217:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226238.320594:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226238.832015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226239.354195:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226239.855989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226240.369062:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226240.879989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226241.392255:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226241.840205:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226241.840214:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226241.840219:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226241.840249:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1834 00000100:00080000:0.0:1761226241.840253:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226241.840259:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226241.840271:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226241.840289:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000243e8b76 x1846777720637440/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226241.840316:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226241.840320:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226241.840465:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720637312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226241.840569:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226241.840576:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720637248:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226241.840600:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720637376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226241.840600:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720637440:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226241.843894:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720637312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226241.852620:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226241.852632:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226241.852643:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720637440:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226241.852652:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720637248:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226241.852667:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720637376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226241.904587:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226242.416138:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226242.927986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226243.440258:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226243.952119:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226244.466536:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226244.977636:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226245.488405:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226246.004281:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226246.514932:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226246.960038:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226246.960046:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226246.960050:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226246.960059:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1840 00000100:00080000:0.0:1761226246.960063:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226246.960072:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:1.0:1761226246.960072:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720637568:192.168.202.114@tcp:400:kworker/u8:2.0 00000008:00100000:0.0:1761226246.960083:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226246.960102:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720637696/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226246.960131:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226246.960151:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226246.960175:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226246.960184:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720637504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226246.960210:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720637632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226246.960501:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720637696:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761226246.965127:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226246.965139:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226246.965150:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720637696:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226246.965258:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720637504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226246.965271:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720637632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226246.965282:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720637568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226247.023974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226247.537177:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226248.047979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226248.560356:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226249.071992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226249.588672:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226250.096003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226250.612134:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226251.119994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226251.632161:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226252.081472:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720637760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226252.082199:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720637824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226252.082363:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226252.082369:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720637888:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226252.082582:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226252.082588:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226252.082591:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226252.082597:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1845 00000100:00080000:1.0:1761226252.082617:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226252.082624:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226252.082635:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226252.082653:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094e95ca3 x1846777720637952/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226252.082712:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720637952:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:1.0:1761226252.082876:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226252.082880:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226252.088398:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720637760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226252.094260:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720637888:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226252.094281:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720637824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226252.094521:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226252.094527:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226252.094550:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720637952:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226252.143973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226252.655998:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226253.167989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226253.680090:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226254.192888:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226254.710901:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226255.216094:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226255.728004:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226256.239982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226256.752160:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226257.200456:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226257.200465:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226257.200469:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226257.200479:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1850 00000100:00080000:0.0:1761226257.200483:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226257.200488:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226257.200499:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226257.200514:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018849446 x1846777720638208/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226257.200538:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226257.200542:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226257.200634:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720638208:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226257.200995:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720638016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226257.201044:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720638144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226257.201292:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226257.212198:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226257.212296:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720638144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226257.217087:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226257.217099:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226257.217109:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720638208:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226257.217375:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720638016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226257.270326:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226257.776015:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226258.288031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226258.800096:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226259.313898:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226259.824095:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226260.335979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226260.848261:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226261.359984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226261.872512:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226262.320061:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226262.320070:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226262.320073:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226262.320084:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1855 00000100:00080000:1.0:1761226262.320088:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226262.320111:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226262.320122:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226262.320138:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720638464/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226262.320163:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226262.320166:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226262.320232:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720638464:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226262.320408:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720638272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226262.320440:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226262.320465:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226262.320471:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:2.0:1761226262.324178:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00100000:0.0:1761226262.330574:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226262.330596:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720638272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226262.330607:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226262.337508:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226262.337521:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226262.337535:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720638464:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226262.386199:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226262.897008:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226263.408059:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226263.922033:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226264.432042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226264.944166:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226265.460354:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226265.968427:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226266.480164:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226266.994492:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226267.440059:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226267.440067:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226267.440071:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226267.440081:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1860 00000100:00080000:1.0:1761226267.440084:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226267.440091:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226267.440103:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226267.440117:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b1564d60 x1846777720638720/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226267.440140:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226267.440143:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226267.440538:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720638528:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226267.440631:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720638656:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226267.440662:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226267.440666:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720638592:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226267.444277:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720638720:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226267.456479:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720638656:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226267.457076:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226267.457082:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226267.457087:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720638720:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226267.457223:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720638592:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226267.457234:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720638528:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226267.504147:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226268.016269:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226268.528353:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226269.041675:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226269.551979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226270.065905:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226270.575978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226271.088148:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226271.600022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226272.116845:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226272.560050:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226272.560059:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226272.560064:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226272.560075:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1865 00000100:00080000:1.0:1761226272.560096:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226272.560103:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226272.560113:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226272.560133:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aa6e6fe7 x1846777720638976/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226272.560161:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226272.560165:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226272.560250:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720638784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226272.560344:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226272.560376:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226272.560382:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226272.560550:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720638976:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226272.567288:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720638784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226272.571704:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226272.571723:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720638912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226272.575195:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226272.575208:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226272.575218:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720638976:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226272.626451:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226273.136212:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226273.651997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226274.163727:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226274.672000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226275.184645:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226275.695987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226276.208102:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226276.720644:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226277.232900:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226277.680456:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226277.680465:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226277.680479:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226277.680490:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1870 00000100:00080000:2.0:1761226277.680494:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226277.680502:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226277.680515:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226277.680532:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007221726f x1846777720639232/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226277.680558:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226277.680562:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226277.680590:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720639040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226277.680660:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720639104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226277.680677:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720639168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226277.680894:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720639232:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226277.687540:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720639104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226277.687574:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226277.687580:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226277.687586:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720639232:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226277.687610:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720639040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226277.687643:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720639168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226277.744253:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226278.256396:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226278.767988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226279.282800:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226279.791981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226280.303993:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226280.816004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226281.328009:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226281.839988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226282.355987:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226282.800052:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226282.800061:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00100000:1.0:1761226282.800066:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720639296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226282.800074:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226282.800082:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1875 00000100:00080000:0.0:1761226282.800085:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226282.800092:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226282.800103:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226282.800119:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720639488/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761226282.800157:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720639424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226282.800162:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226282.800167:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226282.800191:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720639360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226282.800425:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720639488:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226282.824808:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720639360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226282.824834:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226282.824845:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226282.824883:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720639296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226282.824884:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720639488:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226282.824902:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720639424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226282.867062:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226283.382565:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226283.889583:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226284.403086:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226284.915673:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226285.424175:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226285.936067:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226286.448202:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226286.959993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226287.472981:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226287.925730:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226287.925739:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226287.925743:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226287.925754:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1881 00000100:00080000:3.0:1761226287.925758:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226287.925764:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226287.925774:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226287.925789:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720639744/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226287.925814:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226287.925818:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226287.925843:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720639616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226287.925973:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761226287.925980:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720639552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226287.926003:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720639680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226287.926106:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720639744:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226287.932821:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720639616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226287.932879:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720639680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226287.934227:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720639552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226287.934256:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226287.934262:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226287.934266:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720639744:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226287.984043:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226288.496256:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226289.008021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226289.520033:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226290.031993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226290.544192:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226291.055990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226291.568092:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226292.082014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226292.592006:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226293.040739:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226293.040747:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226293.040751:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226293.040761:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1886 00000100:00080000:0.0:1761226293.040765:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226293.040771:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226293.040781:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226293.040798:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d88fcfe6 x1846777720640000/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226293.040824:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226293.040827:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226293.040934:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720640000:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226293.040958:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720639808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226293.041029:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720639936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226293.041057:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226293.041071:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720639872:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226293.049179:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226293.049191:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226293.049201:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720640000:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226293.049710:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720639872:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226293.049723:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720639808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226293.049732:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720639936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226293.104199:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226293.619515:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226294.127987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226294.640210:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226295.152875:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226295.664712:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226296.175984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226296.690022:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226297.200209:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226297.712095:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226298.162154:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226298.162163:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226298.162167:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226298.162178:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1891 00000100:00080000:3.0:1761226298.162181:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226298.162186:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226298.162196:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226298.162210:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035883416 x1846777720640256/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226298.162269:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226298.162273:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226298.162427:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720640064:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226298.163993:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720640256:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226298.169216:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720640128:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226298.169475:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226298.177281:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720640064:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226298.178174:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226298.178182:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226298.178188:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720640256:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226298.185883:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226298.189053:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720640128:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226298.224973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226298.738152:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226299.248397:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226299.763518:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226300.271991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226300.786448:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226301.296504:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226301.808052:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226302.319984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226302.831986:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226303.280886:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226303.280896:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226303.280900:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226303.280912:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1896 00000100:00080000:1.0:1761226303.280916:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226303.280923:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226303.280933:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226303.280950:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720640512/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226303.280990:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226303.280993:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226303.281021:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720640384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226303.281116:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226303.281123:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720640320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226303.281150:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720640448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226303.281592:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720640512:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226303.290602:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720640320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226303.291751:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226303.291758:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226303.291764:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720640512:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226303.292112:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720640448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226303.292126:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720640384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226303.346475:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226303.856093:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226304.367973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226304.880133:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226305.392083:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226305.904753:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226306.415986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226306.932209:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226307.440683:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226307.952038:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226308.400098:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720640576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226308.401145:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226308.402649:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226308.402657:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226308.402661:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226308.402670:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1901 00000100:00080000:3.0:1761226308.402674:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226308.402679:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226308.402690:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226308.402705:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008dec2428 x1846777720640768/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761226308.402761:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720640768:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226308.403761:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226308.403774:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226308.404941:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226308.404949:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761226308.416656:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226308.416667:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226308.416677:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720640768:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226308.416769:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226308.416782:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226308.416941:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720640576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226308.463978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226308.977912:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226309.487995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226310.000127:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226310.511990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226311.024144:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226311.540424:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226312.048220:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226312.561309:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226313.072053:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226313.520057:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226313.520065:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226313.520069:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226313.520081:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1906 00000100:00080000:3.0:1761226313.520096:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226313.520112:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226313.520122:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226313.520136:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abb9f4e4 x1846777720641024/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226313.520159:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226313.520162:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226313.520346:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720640832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226313.520434:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226313.520461:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226313.520466:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640896:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226313.520683:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720641024:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761226313.529170:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1761226313.529180:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226313.529182:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226313.529190:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720641024:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226313.529216:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720640896:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226313.529227:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720640832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226313.583989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226314.096129:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226314.608015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226315.119995:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226315.632134:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226316.144732:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226316.658822:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226317.168095:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226317.681270:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226318.194551:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226318.640133:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720641088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226318.643201:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720641152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226318.648507:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720641088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226318.651436:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226318.651445:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226318.651449:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226318.651459:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1911 00000100:00080000:0.0:1761226318.651463:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226318.651471:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226318.651482:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226318.651499:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720641280/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761226318.651567:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720641280:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761226318.652582:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226318.652590:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226318.652622:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720641216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226318.652715:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720641152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226318.655334:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226318.655345:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226318.655354:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720641280:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226318.655897:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720641216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226318.703965:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226319.216040:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226319.728378:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226320.241222:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226320.751935:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226321.265020:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226321.775986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226322.288115:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226322.799996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226323.312668:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226323.761172:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226323.761180:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226323.761184:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226323.761196:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1916 00000100:00080000:2.0:1761226323.761199:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226323.761216:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226323.761226:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226323.761242:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072dbf25b x1846777720641536/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226323.761267:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226323.761271:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226323.761297:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720641408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226323.761422:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761226323.761428:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720641344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226323.761454:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720641472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226323.761560:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720641536:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761226323.770480:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226323.770494:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226323.770505:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720641536:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226323.770879:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720641344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226323.770897:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720641472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226323.770909:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720641408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226323.825556:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226324.338726:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226324.847976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226325.361106:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226325.871962:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226326.385214:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226326.900805:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226327.408474:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226327.920016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226328.432517:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226328.880139:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720641600:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226328.881871:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226328.881877:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226328.881881:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226328.881889:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1922 00000100:00080000:0.0:1761226328.881893:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226328.881898:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226328.881909:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226328.881923:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000024af3661 x1846777720641792/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226328.881942:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226328.881958:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226328.883024:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720641792:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226328.883359:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720641664:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226328.883397:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226328.883404:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720641728:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226328.885080:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720641600:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226328.889353:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720641664:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226328.889388:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720641728:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226328.891339:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226328.891350:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226328.891358:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720641792:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226328.944110:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226329.457571:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226329.971072:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226330.481307:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226330.992026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226331.505422:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226332.016010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226332.528625:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226333.040704:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226333.552493:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226334.000152:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226334.000160:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226334.000164:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226334.000173:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1927 00000100:00080000:1.0:1761226334.000177:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226334.000197:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226334.000207:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226334.000220:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005d7eada1 x1846777720642048/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226334.000244:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226334.000247:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226334.000275:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720641920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226334.000285:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720642048:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226334.000499:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720641856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226334.000527:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720641984:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226334.018851:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720641920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226334.018890:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226334.018905:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226334.018910:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720642048:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226334.019014:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720641856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226334.019030:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720641984:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226334.067318:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226334.577726:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226335.087998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226335.602630:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226336.111997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226336.624504:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226337.136114:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226337.650106:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226338.163988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226338.672664:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226339.127837:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720642112:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226339.129235:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720642176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226339.130531:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720642240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226339.133223:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226339.133235:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226339.133239:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226339.133249:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1932 00000100:00080000:0.0:1761226339.133253:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226339.133276:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226339.133287:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226339.133314:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000805a30b x1846777720642304/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226339.133389:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720642304:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226339.134417:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720642176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226339.134568:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226339.134573:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761226339.137565:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226339.137579:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226339.137757:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720642304:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226339.137884:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720642240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226339.138698:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720642112:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226339.184555:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226339.696185:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226340.207984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226340.720004:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226341.231987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226341.744064:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226342.258086:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226342.768592:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226343.279983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226343.792065:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226344.240147:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226344.240156:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226344.240160:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226344.240173:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1937 00000100:00100000:2.0:1761226344.240175:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720642368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226344.240178:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 6s 00000100:00080000:3.0:1761226344.240188:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226344.240201:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226344.240218:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000006d85789 x1846777720642560/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226344.240245:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226344.240249:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226344.240273:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720642432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226344.240285:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720642496:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226344.240394:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720642560:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226344.245903:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720642368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226344.246740:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720642432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226344.246811:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720642496:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226344.248779:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226344.248797:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226344.248807:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720642560:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226344.304851:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226344.817154:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226345.327990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226345.840253:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226346.351992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226346.864258:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226347.377415:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226347.890638:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226348.399981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226348.912187:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226349.360110:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226349.360118:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226349.360122:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226349.360133:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1942 00000100:00080000:2.0:1761226349.360137:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1761226349.360143:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226349.360154:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226349.360168:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c987228a x1846777720642816/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226349.360193:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226349.360197:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226349.360227:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720642688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226349.360331:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761226349.360338:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720642624:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226349.360365:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720642752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226349.360455:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720642816:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226349.369593:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720642624:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226349.371781:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720642688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226349.371815:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720642752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226349.373641:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226349.373652:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226349.373675:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720642816:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226349.423988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226349.936299:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226350.447973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226350.960312:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226351.473567:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226351.984342:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226352.495995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226353.009929:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226353.520104:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226354.032210:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226354.480122:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226354.480131:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226354.480136:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226354.480147:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1947 00000100:00080000:0.0:1761226354.480152:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1761226354.480177:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226354.480189:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226354.480205:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720643072/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226354.480230:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226354.480233:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226354.480259:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720642944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226354.480350:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226354.480357:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720642880:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226354.480377:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720643008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226354.481890:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720643072:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226354.484332:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720642880:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226354.487110:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720643008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226354.487131:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720642944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226354.487278:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226354.487283:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226354.487288:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720643072:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226354.544947:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226355.056364:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226355.567968:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226356.080666:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226356.592031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226357.104548:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226357.615987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226358.128026:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226358.640057:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226359.152102:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226359.604050:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226359.604060:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226359.604065:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226359.604076:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1952 00000100:00080000:2.0:1761226359.604081:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761226359.604088:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226359.604100:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226359.604117:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c987228a x1846777720643328/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226359.604142:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226359.604145:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226359.604242:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720643328:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226359.604596:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720643136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226359.604650:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720643264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226359.604676:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226359.604682:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720643200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226359.607261:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761226359.607268:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720643200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226359.607273:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226359.607288:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720643328:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226359.607296:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720643136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226359.607316:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720643264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226359.664140:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226360.179923:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226360.688341:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226361.201233:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226361.714136:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226362.225174:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226362.735993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226363.248464:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226363.760252:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226364.273911:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226364.720144:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226364.720153:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226364.720157:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226364.720169:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1957 00000100:00100000:0.0:1761226364.720170:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720643392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226364.720173:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761226364.720179:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226364.720191:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226364.720208:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005d7eada1 x1846777720643584/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226364.720233:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226364.720236:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226364.720260:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720643456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226364.720266:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720643520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226364.720508:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720643584:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226364.737684:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720643392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226364.740747:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720643456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226364.740851:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226364.740868:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226364.740873:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720643584:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226364.740956:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720643520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226364.784838:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226365.297644:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226365.808032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226366.320651:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226366.835189:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226367.344224:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226367.863276:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226368.368262:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226368.881365:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226369.395739:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226369.840218:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226369.840228:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226369.840232:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226369.840246:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1962 00000100:00080000:0.0:1761226369.840250:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761226369.840258:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226369.840272:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226369.840291:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a99a8bd5 x1846777720643840/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226369.840317:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226369.840321:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226369.840430:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720643840:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226369.840645:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720643712:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226369.840684:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761226369.840690:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720643648:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226369.840736:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720643776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226369.843255:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720643648:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226369.843284:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720643712:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226369.851704:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226369.851714:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226369.851723:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720643840:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226369.852275:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720643776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226369.903995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226370.421152:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226370.928006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226371.441635:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226371.952881:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226372.465800:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226372.978669:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226373.489996:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226373.999988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226374.512067:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226374.960111:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226374.960119:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226374.960123:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226374.960133:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1968 00000100:00080000:2.0:1761226374.960136:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761226374.960143:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226374.960153:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226374.960168:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046bdcfa1 x1846777720644096/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226374.960214:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226374.960218:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226374.960294:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720644096:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226374.960703:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720643904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226374.963075:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720643968:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226374.964318:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720644032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226374.970715:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226374.970729:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226374.970738:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720644096:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226374.970835:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720643968:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226374.970896:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720643904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226374.978187:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720644032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226375.035368:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226375.537478:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226376.048771:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226376.565582:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226377.071981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226377.584106:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226378.095987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226378.609380:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226379.119978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226379.632053:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226380.082064:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226380.082075:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226380.082079:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226380.082091:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1973 00000100:00080000:1.0:1761226380.082105:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761226380.082112:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226380.082124:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226380.082141:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720644352/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226380.082167:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226380.082172:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226380.082201:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720644224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226380.082300:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226380.082306:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720644160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226380.082329:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720644288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226380.082599:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720644352:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761226380.092702:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226380.092714:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226380.092725:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720644352:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226380.092818:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720644288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226380.092885:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720644224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226380.096768:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720644160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226380.144742:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226380.656034:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226381.171751:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226381.680075:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226382.195934:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226382.705070:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226383.216118:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226383.728615:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226384.240008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226384.752058:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226385.200707:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226385.200715:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226385.200720:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226385.200731:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1978 00000100:00080000:2.0:1761226385.200734:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761226385.200740:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226385.200751:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226385.200765:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fab9894 x1846777720644608/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226385.200788:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226385.200801:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226385.200826:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720644416:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226385.200918:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720644544:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226385.200943:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226385.200948:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720644480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226385.201027:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720644608:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226385.209205:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720644480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226385.209224:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720644416:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226385.209235:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720644544:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226385.209346:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226385.209352:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226385.209357:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720644608:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226385.264823:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226385.777672:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226386.289690:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226386.800215:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226387.311998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226387.824083:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226388.336954:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226388.848292:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226389.359989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226389.874598:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226390.320872:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226390.320881:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226390.320885:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226390.320896:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1983 00000100:00080000:1.0:1761226390.320900:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761226390.320906:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226390.320917:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226390.320933:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720644864/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226390.320960:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226390.320963:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226390.321037:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720644672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226390.321129:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720644800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226390.321159:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226390.321164:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720644736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226390.321260:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720644864:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226390.328036:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720644672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226390.328059:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720644800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226390.329699:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226390.329709:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226390.329718:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720644864:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226390.329999:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720644736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226390.384288:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226390.896347:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226391.408003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226391.920109:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226392.432294:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226392.945056:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226393.456000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226393.968381:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226394.479989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226394.992262:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226395.440111:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226395.440118:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226395.440122:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226395.440132:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1988 00000100:00080000:1.0:1761226395.440136:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761226395.440143:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226395.440153:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226395.440169:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abc685ce x1846777720645120/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226395.440207:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226395.440210:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226395.440326:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720645120:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226395.443780:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720644928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226395.443909:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720645056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226395.443943:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226395.443949:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720644992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226395.445069:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226395.445077:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226395.445083:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720645120:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226395.450430:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720644992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226395.450451:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720644928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226395.450461:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720645056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226395.509693:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226396.016916:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226396.528057:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226397.040167:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226397.551990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226398.067047:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226398.576005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226399.088058:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226399.599965:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226400.113083:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226400.561636:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226400.561644:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226400.561649:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226400.561662:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1993 00000100:00080000:3.0:1761226400.561666:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226400.561694:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226400.561706:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226400.561724:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000074fca4bf x1846777720645376/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226400.561753:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226400.561757:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226400.561806:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720645376:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226400.561836:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720645184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226400.562100:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720645248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226400.562421:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720645312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226400.568510:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720645184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226400.571051:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226400.571063:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226400.571073:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720645376:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226400.571783:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720645312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226400.572963:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720645248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226400.624847:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226401.136707:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226401.650197:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226402.160206:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226402.685493:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226403.183985:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226403.696741:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226404.208134:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226404.719981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226405.232590:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226405.680071:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226405.680080:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226405.680084:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226405.680095:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 1998 00000100:00080000:3.0:1761226405.680099:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226405.680106:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226405.680119:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226405.680136:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720645632/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226405.680242:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720645632:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761226405.680599:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226405.680681:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226405.680762:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720645440:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226405.681009:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720645568:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226405.681047:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226405.681053:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720645504:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226405.682564:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226405.682572:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226405.682578:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720645632:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226405.695838:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720645440:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226405.700793:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720645504:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226405.700816:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720645568:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226405.744055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226406.257153:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226406.776428:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226407.280002:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226407.791995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226408.304249:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226408.815987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226409.328173:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226409.844877:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226410.353053:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226410.801394:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226410.801404:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226410.801408:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226410.801421:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2003 00000100:00080000:3.0:1761226410.801425:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226410.801432:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226410.801446:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226410.801462:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720645888/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226410.801489:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226410.801493:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226410.801595:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720645888:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226410.802519:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720645696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226410.802587:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720645824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226410.802618:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761226410.802625:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720645760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226410.807501:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720645760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226410.807522:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720645824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226410.814939:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226410.814952:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226410.814962:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720645888:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226410.815804:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720645696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226410.863980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226411.376468:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226411.896133:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226412.401377:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226412.912074:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226413.424247:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226413.935999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226414.448110:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226414.960768:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226415.472202:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226415.922330:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226415.922339:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226415.922344:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226415.922355:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2009 00000100:00080000:0.0:1761226415.922358:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761226415.922365:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226415.922374:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226415.922390:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720646144/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226415.922411:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226415.922431:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226415.922459:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720646016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226415.922551:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226415.922558:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720645952:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226415.922581:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720646080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226415.922889:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720646144:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226415.929187:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720646080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226415.929698:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720646016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226415.932173:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720645952:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:3.0:1761226415.934536:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:3.0:1761226415.937664:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226415.937674:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226415.937685:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720646144:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226415.984003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226416.496136:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226417.008448:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226417.522203:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226418.031978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226418.544261:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226419.059263:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226419.568322:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226420.080001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226420.593536:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226421.041917:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720646208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226421.046209:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226421.046219:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226421.046224:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226421.046236:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2014 00000100:00080000:0.0:1761226421.046239:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761226421.046246:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226421.046258:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226421.046274:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720646400/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226421.046336:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720646400:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761226421.046703:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226421.046706:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226421.046813:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720646272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226421.046900:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226421.046907:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720646336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226421.047589:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720646208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226421.054168:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226421.054181:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226421.054194:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720646400:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226421.055100:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720646336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226421.055114:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720646272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226421.106215:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226421.616081:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226422.127979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226422.640113:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226423.152362:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226423.664128:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226424.176022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226424.688060:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226425.200004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226425.714622:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226426.161530:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226426.161538:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226426.161542:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226426.161552:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2019 00000100:00080000:3.0:1761226426.161556:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226426.161562:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226426.161571:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226426.161587:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000006d85789 x1846777720646656/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226426.161615:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226426.161619:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226426.161775:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720646656:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226426.161966:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720646464:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226426.162004:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720646592:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226426.162028:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226426.162033:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720646528:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226426.171135:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226426.171148:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226426.171159:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720646656:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226426.171268:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720646528:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226426.171280:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720646464:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226426.171289:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720646592:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226426.223994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226426.738554:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226427.248204:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226427.761084:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226428.271989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226428.784111:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226429.314452:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226429.808119:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226430.323038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226430.832114:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226431.280220:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226431.280228:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226431.280232:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226431.280244:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2024 00000100:00080000:1.0:1761226431.280247:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761226431.280252:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226431.280262:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226431.280278:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dc5f36b1 x1846777720646912/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226431.280349:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720646912:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226431.280485:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720646720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226431.280522:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720646848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226431.280551:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226431.280557:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720646784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226431.283488:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226431.283500:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226431.284099:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720646784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226431.284177:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720646720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226431.284188:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720646848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226431.284270:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226431.284275:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226431.284280:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720646912:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226431.345560:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226431.856220:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226432.368015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226432.880212:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226433.393485:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226433.904103:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226434.415990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226434.928144:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226435.440095:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226435.954155:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226436.402809:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720646976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226436.404604:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226436.407404:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226436.407413:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226436.407417:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226436.407427:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2029 00000100:00080000:3.0:1761226436.407431:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226436.407437:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226436.407447:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226436.407461:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d56d40dd x1846777720647168/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226436.407487:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226436.407490:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226436.407514:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226436.407529:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647104:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226436.408042:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720647168:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226436.409283:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720646976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226436.409551:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226436.415977:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226436.416051:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226436.416064:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720647168:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226436.416911:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647104:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226436.463987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226436.977259:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226437.487984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226438.000404:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226438.511973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226439.028222:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226439.537971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226440.048200:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226440.559991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226441.072298:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226441.520109:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226441.520118:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226441.520122:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226441.520150:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2034 00000100:00080000:3.0:1761226441.520154:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226441.520161:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226441.520172:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226441.520187:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000341ad5cd x1846777720647424/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226441.520219:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226441.520223:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226441.520343:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720647424:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226441.520517:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720647232:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226441.520553:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647360:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226441.520577:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226441.520582:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226441.523298:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226441.523316:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720647232:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226441.523343:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647360:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226441.524085:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226441.524092:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226441.524098:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720647424:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226441.584001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226442.096097:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226442.607979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226443.121129:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226443.637479:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226444.144551:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226444.657130:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226445.177245:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226445.679938:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226446.193426:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226446.641542:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720647488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226446.643215:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720647552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226446.645006:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720647488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226446.647715:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226446.647724:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226446.647728:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226446.647740:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2039 00000100:00080000:0.0:1761226446.647744:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761226446.647984:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226446.647998:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226446.648032:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003e02fcda x1846777720647680/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226446.648108:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720647680:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761226446.649359:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226446.649368:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226446.649394:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226446.649422:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720647616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226446.649995:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720647552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226446.657692:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720647616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226446.659204:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226446.659214:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226446.659222:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720647680:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226446.704003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226447.216938:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226447.729035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226448.241946:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226448.752042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226449.267331:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226449.775989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226450.288071:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226450.799980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226451.312196:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226451.760178:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226451.760188:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226451.760193:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226451.760203:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2044 00000100:00080000:2.0:1761226451.760207:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761226451.760214:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226451.760225:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226451.760241:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003ba27032 x1846777720647936/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226451.760266:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226451.760270:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226451.760292:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226451.760395:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761226451.760402:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720647744:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226451.760446:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226451.763179:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720647936:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226451.766024:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226451.766043:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720647808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226451.766722:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720647744:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226451.766812:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226451.766818:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226451.766822:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720647936:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226451.823974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226452.338778:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226452.847974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226453.360014:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226453.872654:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226454.389923:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226454.896509:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226455.408099:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226455.920900:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226456.431988:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226456.880022:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1761226456.880022:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720648000:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226456.880030:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226456.880034:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226456.880041:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2050 00000100:00080000:2.0:1761226456.880044:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761226456.880051:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226456.880063:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226456.880081:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720648192/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226456.880102:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226456.880105:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226456.880139:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720648064:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226456.880150:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720648128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226456.880291:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720648192:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226456.883809:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720648064:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226456.884058:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720648000:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226456.884078:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720648128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226456.885463:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226456.885472:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226456.885480:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720648192:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226456.944535:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226457.456274:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226457.968880:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226458.480064:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226458.992984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226459.504072:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226460.015981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226460.535091:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226461.039987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226461.552661:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226462.001646:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226462.001653:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226462.001657:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226462.001667:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2055 00000100:00080000:0.0:1761226462.001671:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761226462.001677:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226462.001687:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226462.001702:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018849446 x1846777720648448/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226462.001726:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226462.001730:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226462.001777:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720648320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226462.001914:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226462.001920:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720648256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226462.001958:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720648384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226462.002478:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720648448:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226462.009129:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720648256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226462.009153:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720648384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226462.009165:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720648320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226462.013830:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226462.013891:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226462.013902:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720648448:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226462.063915:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226462.576020:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226463.088823:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226463.602302:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226464.116736:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226464.624240:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226465.136069:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226465.663958:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226466.160023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226466.672279:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226467.120075:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226467.120085:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226467.120089:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226467.120101:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2060 00000100:00080000:0.0:1761226467.120104:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1761226467.120109:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720648576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226467.120110:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226467.120121:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226467.120138:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720648704/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226467.120163:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226467.120167:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226467.120203:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720648512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226467.120234:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720648640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226467.120405:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720648704:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226467.125768:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720648576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226467.126006:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226467.126014:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226467.126020:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720648704:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226467.126066:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720648512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226467.126078:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720648640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226467.183968:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226467.698774:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226468.208652:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226468.722873:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226469.231977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226469.744042:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226470.255946:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226470.768233:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226471.280545:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226471.792826:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226472.240061:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226472.240070:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226472.240073:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226472.240083:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2065 00000100:00080000:0.0:1761226472.240087:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761226472.240093:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226472.240103:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226472.240118:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720648960/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226472.240194:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720648960:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226472.240323:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720648768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226472.240355:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720648896:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226472.240381:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226472.240403:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720648832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226472.240725:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226472.240729:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226472.254515:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720648832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226472.254541:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720648768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226472.254549:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720648896:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226472.257728:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226472.257739:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226472.257750:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720648960:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226472.304065:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226472.824718:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226473.327991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226473.840736:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226474.352003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226474.864319:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226475.375974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226475.888421:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226476.401084:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226476.912700:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226477.360143:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226477.360154:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226477.360158:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226477.360174:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2070 00000100:00080000:0.0:1761226477.360198:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761226477.360206:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226477.360222:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226477.360242:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fe9fa97 x1846777720649216/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226477.360266:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226477.360270:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226477.360309:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720649088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226477.360393:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720649216:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226477.360444:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761226477.360455:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720649024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226477.360490:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720649152:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226477.364309:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720649024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226477.364343:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720649152:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226477.364357:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720649088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226477.365027:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226477.365037:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226477.365044:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720649216:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226477.424017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226477.936184:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226478.448033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226478.962181:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226479.473013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226479.984228:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226480.496243:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226481.008481:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226481.520020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226482.039676:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226482.480069:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226482.480077:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226482.480082:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226482.480092:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2075 00000100:00080000:1.0:1761226482.480096:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761226482.480102:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226482.480113:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226482.480127:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ef613dc x1846777720649472/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226482.480186:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720649472:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226482.480325:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720649280:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226482.480359:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720649408:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226482.480428:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226482.480435:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720649344:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761226482.481907:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226482.481916:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761226482.492379:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226482.492401:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226482.492412:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720649472:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:1.0:1761226482.494019:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720649280:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226482.504171:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720649344:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226482.504192:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720649408:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226482.544022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226483.060917:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226483.568018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226484.086480:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226484.591966:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226485.104463:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226485.615999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226486.130393:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226486.640038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226487.152175:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226487.600186:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720649536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226487.600710:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226487.600717:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720649600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226487.601827:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226487.601834:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226487.601838:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226487.601845:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2080 00000100:00080000:0.0:1761226487.601848:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761226487.601867:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226487.601878:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226487.601894:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000525c9db8 x1846777720649728/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226487.601959:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720649728:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226487.602731:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720649664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226487.603687:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226487.603694:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226487.609606:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720649600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226487.609627:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720649536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226487.614049:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226487.614059:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226487.614068:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720649728:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226487.615544:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720649664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226487.666595:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226488.176093:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226488.688840:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226489.201786:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226489.712498:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226490.224032:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226490.735927:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226491.249716:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226491.766829:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226492.272049:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226492.726633:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226492.726642:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226492.726646:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226492.726658:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2085 00000100:00080000:1.0:1761226492.726662:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226492.726669:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226492.726678:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226492.726694:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d0d44bb x1846777720649984/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226492.726710:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226492.726713:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226492.726737:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720649856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226492.726752:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720649984:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226492.726839:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761226492.726846:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720649792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226492.728220:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720649920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226492.735200:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1761226492.735204:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720649792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226492.735211:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226492.735218:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720649984:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226492.735224:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720649920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226492.735237:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720649856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226492.784001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226493.296044:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226493.807982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226494.322106:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226494.831975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226495.344133:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226495.857268:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226496.368279:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226496.880202:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226497.394162:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226497.840102:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226497.840111:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226497.840114:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226497.840125:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2090 00000100:00080000:1.0:1761226497.840129:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226497.840134:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226497.840144:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226497.840162:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d0d44bb x1846777720650240/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226497.840184:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226497.840187:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226497.840274:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720650048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226497.840390:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720650176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226497.840418:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226497.840422:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720650112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226497.840507:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720650240:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761226497.846668:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226497.846680:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226497.846690:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720650240:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226497.847137:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720650176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226497.847160:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720650112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226497.847169:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720650048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226497.903982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226498.419930:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226498.927978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226499.440962:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226499.951972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226500.464136:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226500.975985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226501.492646:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226502.002897:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226502.512160:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226502.960210:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226502.960217:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226502.960221:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226502.960231:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2096 00000100:00080000:2.0:1761226502.960235:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226502.960247:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226502.960258:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226502.960272:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000290da3b x1846777720650496/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226502.960367:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720650496:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226502.960381:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720650304:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226502.960459:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650432:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226502.960495:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226502.960502:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650368:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226502.962054:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226502.962064:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226502.965324:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650368:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226502.965345:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650432:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226502.966989:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720650304:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226502.967090:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226502.967095:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226502.967100:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720650496:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226503.024280:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226503.536943:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226504.047983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226504.560103:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226505.071978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226505.584220:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226506.096046:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226506.610234:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226507.119989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226507.632167:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226508.081769:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761226508.081783:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720650560:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226508.081790:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226508.081795:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226508.081801:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2101 00000100:00080000:3.0:1761226508.081804:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226508.081810:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226508.081820:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226508.081836:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720650752/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226508.081882:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226508.081885:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226508.081894:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720650688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226508.081904:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650624:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226508.083067:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720650752:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226508.084639:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720650560:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226508.085782:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720650688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226508.087029:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226508.087053:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226508.087061:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720650752:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226508.087359:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650624:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226508.146346:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226508.656159:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226509.168000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226509.680051:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226510.192021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226510.704900:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226511.215989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226511.728529:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226512.239980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226512.756393:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226513.200049:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720650816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226513.201883:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226513.201891:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226513.201895:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226513.201904:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2106 00000100:00080000:3.0:1761226513.201907:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226513.201913:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226513.201923:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226513.201938:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a6d0bfc0 x1846777720651008/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226513.201968:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226513.201971:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226513.201995:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650944:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226513.202077:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226513.202084:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226513.202458:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720651008:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226513.209744:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226513.209764:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720650944:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226513.209774:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720650816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226513.211643:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226513.211653:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226513.211663:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720651008:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226513.282920:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226513.776168:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226514.287975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226514.800352:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226515.313484:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226515.824086:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226516.336421:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226516.848361:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226517.359967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226517.872527:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226518.320312:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226518.320319:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226518.320323:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226518.320333:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2111 00000100:00080000:2.0:1761226518.320336:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226518.320343:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226518.320353:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226518.320369:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec5c4aea x1846777720651264/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226518.320396:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226518.320400:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226518.320402:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720651072:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226518.320438:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720651136:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226518.320494:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720651200:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226518.320560:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720651264:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226518.323776:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720651072:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226518.326431:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720651200:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226518.326517:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720651136:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761226518.326591:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226518.326597:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226518.326601:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720651264:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226518.383983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226518.897958:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226519.408169:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226519.920109:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226520.433120:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226520.945218:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226521.455979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226521.968975:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226522.481568:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226522.993013:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226523.441974:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226523.441982:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226523.441986:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226523.441999:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2116 00000100:00100000:1.0:1761226523.441999:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720651328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226523.442003:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226523.442012:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226523.442023:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226523.442042:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000525c9db8 x1846777720651520/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226523.442071:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226523.442074:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226523.442111:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720651456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226523.442146:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226523.442153:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720651392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226523.442195:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720651520:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226523.447569:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720651328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226523.449761:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720651392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226523.449780:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720651456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226523.452769:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226523.452796:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226523.452806:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720651520:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226523.504028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226524.016375:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226524.527987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226525.042063:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226525.552623:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226526.067359:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226526.576583:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226527.090058:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226527.600017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226528.112078:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226528.560599:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720651584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226528.561571:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226528.561583:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720651648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226528.561984:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226528.561990:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226528.561994:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226528.562001:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2121 00000100:00080000:0.0:1761226528.562004:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226528.562010:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226528.562021:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226528.562035:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b0db11ae x1846777720651776/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226528.562074:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226528.562078:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226528.562119:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226528.562129:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720651712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226528.562621:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720651776:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226528.566543:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720651712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226528.566631:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720651584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226528.572046:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720651648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226528.572551:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226528.572557:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226528.572562:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720651776:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226528.627259:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226529.136587:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226529.648046:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226530.160009:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226530.672312:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226531.184094:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226531.696026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226532.209416:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226532.721245:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226533.232550:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226533.680111:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226533.680120:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226533.680123:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226533.680133:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2126 00000100:00080000:1.0:1761226533.680136:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226533.680142:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226533.680151:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226533.680181:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dc5f36b1 x1846777720652032/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226533.680206:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226533.680210:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226533.681710:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720652032:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226533.683495:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720651840:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226533.683766:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720651904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226533.687484:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720651968:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226533.688276:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720651904:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226533.688398:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226533.688403:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226533.688414:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720652032:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226533.699068:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720651840:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226533.701295:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720651968:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226533.744373:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226534.256461:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226534.768045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226535.281075:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226535.793891:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226536.306763:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226536.816697:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226537.328362:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226537.842763:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226538.353960:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226538.805736:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226538.805744:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226538.805748:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226538.805758:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2131 00000100:00080000:1.0:1761226538.805762:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226538.805768:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226538.805778:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226538.805794:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720652288/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226538.805844:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720652160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226538.805844:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226538.805848:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226538.805894:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720652096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226538.805981:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720652224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226538.806343:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720652288:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226538.813229:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720652160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226538.815501:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720652224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226538.819454:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226538.819464:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226538.819474:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720652288:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226538.819561:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720652096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226538.863994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226539.377698:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226539.887977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226540.400135:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226540.912040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226541.424495:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226541.937593:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226542.448285:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226542.961526:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226543.475456:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226543.920328:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720652352:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226543.921777:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226543.921784:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226543.921788:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226543.921796:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2137 00000100:00080000:2.0:1761226543.921800:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226543.921805:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226543.921830:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226543.921845:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f5134a53 x1846777720652544/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226543.921880:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226543.921883:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226543.921907:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720652544:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226543.921989:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720652416:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226543.922018:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226543.922023:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720652480:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226543.922562:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720652352:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226543.929579:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720652416:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226543.929615:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720652480:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226543.930608:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226543.930616:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226543.930622:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720652544:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226543.986214:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226544.496302:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226545.007982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226545.520033:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226546.031992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226546.544085:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226547.058188:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226547.568115:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226548.081573:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226548.592619:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226549.040180:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226549.040189:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226549.040193:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226549.040204:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2142 00000100:00080000:1.0:1761226549.040208:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761226549.040214:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720652608:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:1.0:1761226549.040215:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226549.040229:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226549.040247:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720652800/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226549.040272:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226549.040275:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226549.040311:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720652736:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226549.040321:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720652672:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226549.040684:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720652800:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226549.044071:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720652672:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226549.044071:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720652608:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226549.044098:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720652736:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226549.046764:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226549.046774:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226549.046784:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720652800:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226549.104009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226549.616038:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226550.132564:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226550.641445:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226551.152006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226551.664262:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226552.175956:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226552.688048:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226553.199999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226553.712070:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226554.160942:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720652864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226554.161263:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226554.161270:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720652928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226554.161496:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226554.161501:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226554.161522:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226554.161528:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2147 00000100:00080000:0.0:1761226554.161531:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226554.161537:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226554.161547:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226554.161562:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720653056/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226554.161624:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720653056:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226554.163344:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720652992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226554.165906:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226554.165918:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226554.167582:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720652864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226554.172761:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226554.172773:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226554.172784:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720653056:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226554.172906:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720652992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226554.172919:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720652928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226554.225822:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226554.736291:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226555.247980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226555.761769:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226556.271992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226556.785353:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226557.295991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226557.808115:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226558.320000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226558.832895:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226559.280111:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226559.280119:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226559.280123:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226559.280133:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2152 00000100:00080000:2.0:1761226559.280137:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226559.280143:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226559.280152:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226559.280165:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720653312/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226559.280191:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226559.280195:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226559.280228:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720653184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226559.280232:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720653312:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226559.280304:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720653120:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226559.280336:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720653248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226559.283646:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226559.283660:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226559.283668:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720653312:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226559.287633:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720653248:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226559.288713:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720653184:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226559.289104:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720653120:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226559.343994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226559.855988:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226560.367986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226560.880534:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226561.392648:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226561.904202:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226562.417480:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226562.928106:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226563.439996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226563.952120:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226564.400209:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226564.400217:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226564.400221:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226564.400231:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2157 00000100:00080000:1.0:1761226564.400235:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226564.400241:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226564.400251:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226564.400266:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000158429d4 x1846777720653568/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226564.400289:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226564.400293:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226564.400337:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720653376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226564.400420:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720653504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226564.400453:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226564.400458:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720653440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226564.400711:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720653568:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226564.405386:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720653376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226564.407327:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720653504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226564.408632:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720653440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226564.408808:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226564.408813:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226564.408818:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720653568:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226564.463994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226564.976136:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226565.491683:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226566.000126:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226566.512005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226567.024389:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226567.536526:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226568.048101:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226568.559965:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226569.072060:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226569.520947:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226569.520956:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226569.520961:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226569.520972:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2162 00000100:00080000:1.0:1761226569.520976:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226569.520982:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226569.520992:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226569.521006:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720653824/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226569.521032:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226569.521035:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226569.521150:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720653632:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226569.521245:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720653760:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226569.521278:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226569.521284:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720653696:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226569.523382:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720653824:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226569.525037:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720653696:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226569.527923:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720653632:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226569.529314:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720653760:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:02020000:2.0:1761226569.532362:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:2.0:1761226569.546743:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226569.546756:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226569.546769:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720653824:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226569.583966:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226570.096104:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226570.607968:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226571.120555:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226571.632039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226572.144757:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226572.656018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226573.171927:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226573.679990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226574.194580:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226574.640061:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226574.640067:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226574.640071:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226574.640081:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2167 00000100:00080000:1.0:1761226574.640084:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226574.640105:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226574.640114:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226574.640127:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720654080/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226574.640149:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226574.640152:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226574.640238:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720653888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226574.640320:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720654016:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226574.640345:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226574.640357:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720653952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226574.645350:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720654080:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226574.646812:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720653952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226574.646832:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720653888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226574.646842:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720654016:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226574.654952:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226574.654963:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226574.654971:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720654080:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226574.704934:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226575.218799:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226575.727967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226576.240036:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226576.751963:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226577.264123:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226577.780545:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226578.288146:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226578.799957:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226579.311987:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226579.760618:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226579.760627:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226579.760631:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226579.760640:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2172 00000100:00080000:0.0:1761226579.760643:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226579.760649:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226579.760658:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226579.760679:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720654336/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226579.760747:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720654336:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761226579.761551:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226579.761556:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226579.761577:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720654208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226579.761618:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761226579.761641:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720654144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226579.761662:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720654272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226579.770987:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720654272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226579.771008:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720654208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226579.771286:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226579.771292:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226579.771297:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720654336:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226579.771408:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720654144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226579.823985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226580.338434:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226580.848020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226581.360155:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226581.872064:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226582.386412:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226582.895976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226583.408187:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226583.921491:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226584.432253:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226584.880114:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226584.880123:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226584.880126:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226584.880137:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2178 00000100:00080000:0.0:1761226584.880142:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226584.880148:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226584.880159:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226584.880174:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7988e14 x1846777720654592/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226584.880199:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226584.880203:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226584.880276:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720654464:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226584.880406:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761226584.880412:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720654400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226584.880435:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720654528:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226584.880519:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720654592:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226584.886388:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720654400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226584.891188:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720654528:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226584.891210:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720654464:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226584.891304:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226584.891310:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226584.891316:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720654592:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226584.944332:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226585.457320:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226585.968374:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226586.480267:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226586.992024:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226587.504097:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226588.016271:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226588.529487:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226589.040040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226589.552154:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226590.000069:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226590.000078:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226590.000083:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226590.000095:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2183 00000100:00080000:0.0:1761226590.000098:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226590.000104:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226590.000114:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226590.000128:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720654848/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226590.000158:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226590.000162:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226590.000308:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720654656:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226590.000409:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720654848:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226590.000442:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720654784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226590.000515:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226590.000523:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720654720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226590.003397:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720654720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226590.003419:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720654656:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226590.003431:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720654784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226590.013845:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226590.013902:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226590.013914:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720654848:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226590.064244:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226590.579311:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226591.088004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226591.603450:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226592.112236:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226592.624111:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226593.138478:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226593.648161:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226594.160032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226594.672007:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226595.120584:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226595.120591:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226595.120595:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226595.120607:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2188 00000100:00080000:0.0:1761226595.120611:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226595.120632:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226595.120644:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226595.120888:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a99a8bd5 x1846777720655104/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226595.120944:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720655104:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226595.121096:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720654912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226595.121131:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720655040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226595.121161:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226595.121167:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720654976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226595.121525:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226595.121530:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761226595.128982:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226595.128999:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226595.129010:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720655104:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226595.130773:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720654912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226595.131650:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720654976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226595.131715:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720655040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226595.184069:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226595.696799:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226596.208036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226596.720132:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226597.233283:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226597.744981:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226598.256048:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226598.768094:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226599.280444:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226599.792117:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226600.240466:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226600.240474:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226600.240478:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226600.240491:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2193 00000100:00080000:3.0:1761226600.240496:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226600.240504:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226600.240517:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226600.240536:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd946368 x1846777720655360/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226600.240581:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226600.240585:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226600.240614:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720655232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226600.240648:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720655168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226600.240711:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720655360:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226600.240731:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720655296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226600.248717:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720655232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226600.261816:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226600.261831:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226600.261843:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720655360:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226600.262278:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720655168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226600.262299:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720655296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226600.305131:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226600.816888:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226601.331348:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226601.842778:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226602.352085:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226602.864274:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226603.375991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226603.888281:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226604.399992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226604.912172:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226605.360360:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226605.360369:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226605.360374:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226605.360387:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2198 00000100:00080000:2.0:1761226605.360391:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226605.360398:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226605.360411:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226605.360428:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006be78503 x1846777720655616/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226605.360456:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226605.360460:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226605.360561:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720655616:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226605.360729:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720655424:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226605.360766:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720655552:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226605.360792:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226605.360797:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720655488:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226605.365654:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226605.365670:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226605.365683:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720655616:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226605.370050:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720655424:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226605.379515:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720655488:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226605.379539:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720655552:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226605.424055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226605.936188:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226606.455922:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226606.965500:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226607.472043:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226607.984300:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226608.496028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226609.008166:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226609.520588:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226610.033732:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226610.480573:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226610.480581:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226610.480601:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226610.480611:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2203 00000100:00080000:2.0:1761226610.480614:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226610.480619:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226610.480629:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226610.480644:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b3214ada x1846777720655872/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226610.480693:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720655872:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226610.480749:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720655680:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226610.480790:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720655808:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226610.480821:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226610.480836:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720655744:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226610.482510:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226610.482519:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761226610.486240:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226610.486252:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226610.486263:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720655872:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226610.492209:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720655744:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226610.492230:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720655680:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226610.492242:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720655808:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226610.543979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226611.056167:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226611.568552:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226612.080251:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226612.591998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226613.104033:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226613.615990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226614.128164:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226614.641282:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226615.152584:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226615.600195:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226615.600204:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226615.600208:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226615.600219:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2208 00000100:00080000:3.0:1761226615.600222:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226615.600228:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226615.600239:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226615.600253:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c2618360 x1846777720656128/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226615.600281:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226615.600284:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226615.600311:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720656000:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226615.600413:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761226615.600419:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720655936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226615.600443:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720656064:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226615.600687:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720656128:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226615.614248:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720655936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226615.614273:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720656064:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226615.614284:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720656000:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226615.614405:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226615.614411:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226615.614415:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720656128:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226615.663973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226616.179000:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226616.687976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226617.200410:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226617.711995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226618.224035:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226618.735985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226619.249246:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226619.759999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226620.275192:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226620.722005:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720656192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226620.723541:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720656256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226620.724639:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720656192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226620.724806:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226620.724812:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226620.724816:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226620.724821:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2213 00000100:00080000:2.0:1761226620.724824:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226620.724830:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226620.724839:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226620.724879:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006bca6691 x1846777720656384/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226620.724895:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226620.724899:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226620.724917:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226620.724927:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720656320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226620.725182:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720656384:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226620.731129:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720656256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226620.743361:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720656320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226620.743435:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226620.743441:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226620.743446:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720656384:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226620.784001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226621.296089:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226621.808192:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226622.323414:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226622.832708:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226623.344128:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226623.855997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226624.368038:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226624.880047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226625.392007:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226625.840060:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226625.840068:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226625.840074:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226625.840085:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2218 00000100:00080000:2.0:1761226625.840088:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226625.840095:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226625.840105:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226625.840121:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720656640/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226625.840146:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226625.840150:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226625.840241:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720656640:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226625.840592:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720656448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226625.840645:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720656576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226625.840671:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226625.840677:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720656512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226625.847578:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226625.847593:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226625.847604:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720656640:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226625.847660:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720656512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226625.847911:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720656448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226625.847938:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720656576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226625.905644:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226626.417427:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226626.927991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226627.442820:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226627.952035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226628.464841:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226628.975994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226629.491891:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226630.000020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226630.516109:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226630.960152:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226630.960162:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226630.960167:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761226630.960174:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720656704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226630.960178:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2224 00000100:00080000:0.0:1761226630.960184:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226630.960191:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226630.960203:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226630.960222:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720656896/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226630.960247:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226630.960251:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226630.960269:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720656832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226630.960274:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720656768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226630.960602:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720656896:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226630.969505:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720656704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226630.970313:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226630.970319:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226630.970329:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720656896:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226630.970712:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720656768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226630.970744:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720656832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226631.024405:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226631.537008:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226632.049807:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226632.560105:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226633.071990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226633.584653:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226634.096015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226634.607999:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226635.119986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226635.632596:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226636.080740:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226636.080748:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226636.080753:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226636.080764:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2229 00000100:00080000:0.0:1761226636.080768:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226636.080774:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226636.080785:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226636.080800:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000805a30b x1846777720657152/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226636.080831:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226636.080835:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226636.080877:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720657024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226636.080974:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226636.080982:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720656960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226636.081005:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720657088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226636.083229:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720657152:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226636.087481:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720656960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226636.087504:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720657088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226636.087514:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720657024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226636.089050:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226636.089059:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226636.089066:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720657152:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226636.144242:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226636.656633:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226637.168006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226637.681068:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226638.192894:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226638.704091:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226639.216171:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226639.729005:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226640.239985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226640.752264:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226641.200059:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226641.200068:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226641.200073:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226641.200084:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2234 00000100:00080000:0.0:1761226641.200088:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226641.200094:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226641.200104:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226641.200121:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720657408/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226641.200148:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226641.200152:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226641.200318:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720657280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226641.200511:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720657408:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226641.200749:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226641.200756:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720657216:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226641.202568:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720657344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226641.205950:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226641.205962:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226641.205972:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720657408:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226641.206470:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720657280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226641.207879:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720657216:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226641.207897:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720657344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226641.264602:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226641.777216:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226642.287988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226642.801156:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226643.311981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226643.824378:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226644.336003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226644.848087:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226645.363394:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226645.872766:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226646.320055:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226646.320064:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226646.320069:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226646.320079:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2239 00000100:00080000:0.0:1761226646.320082:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226646.320090:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226646.320100:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226646.320116:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720657664/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226646.320141:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226646.320144:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226646.320172:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720657536:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226646.320270:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226646.320277:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720657472:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226646.320302:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720657600:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226646.320841:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720657664:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226646.327385:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720657536:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226646.329114:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720657472:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:1.0:1761226646.329134:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720657600:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226646.333035:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226646.333048:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226646.333058:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720657664:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226646.389168:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226646.896941:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226647.408187:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226647.920463:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226648.432222:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226648.946005:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226649.456031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226649.968774:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226650.482198:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226650.992152:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226651.440738:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226651.440764:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226651.440768:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226651.440781:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2244 00000100:00080000:3.0:1761226651.440785:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226651.440793:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226651.440806:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226651.440823:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f9712cd6 x1846777720657920/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226651.440847:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226651.440850:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226651.440926:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720657792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226651.441025:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761226651.441031:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720657728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226651.441050:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720657856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226651.441997:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720657920:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761226651.454297:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226651.454326:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226651.454336:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720657920:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226651.455152:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720657728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226651.455172:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720657856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226651.455184:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720657792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226651.504175:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226652.018730:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226652.530592:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226653.040188:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226653.552045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226654.067845:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226654.576029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226655.089594:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226655.600085:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226656.113580:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226656.560118:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226656.560126:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226656.560131:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226656.560144:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2249 00000100:00080000:3.0:1761226656.560166:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226656.560173:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226656.560188:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226656.560208:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092860b64 x1846777720658176/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226656.560237:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226656.560240:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226656.560278:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226656.560399:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761226656.560406:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720657984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226656.560433:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226656.560552:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720658176:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226656.567477:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226656.568392:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226656.568404:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226656.568414:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720658176:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226656.568536:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720657984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226656.568552:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226656.624078:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226657.136704:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226657.647990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226658.160705:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226658.672021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226659.187969:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226659.696027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226660.208070:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226660.720105:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226661.237462:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226661.680041:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226661.680049:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226661.680052:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226661.680063:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2254 00000100:00080000:3.0:1761226661.680066:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226661.680072:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226661.680082:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226661.680113:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8803538 x1846777720658432/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226661.680193:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720658432:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226661.680722:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720658240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226661.680768:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226661.680798:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761226661.680804:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226661.681887:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226661.681896:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226661.684754:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226661.684770:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720658240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226661.684779:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226661.686061:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226661.686069:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226661.686076:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720658432:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226661.743987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226662.256217:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226662.768050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226663.280008:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226663.791994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226664.305581:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226664.815985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226665.328153:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226665.840793:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226666.353843:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226666.801633:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226666.801642:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226666.801647:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226666.801661:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2259 00000100:00080000:2.0:1761226666.801665:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226666.801690:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226666.801702:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226666.801720:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072dbf25b x1846777720658688/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226666.801742:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226666.801745:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226666.801773:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226666.801906:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761226666.801914:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720658496:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226666.801941:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226666.802888:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720658688:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226666.806085:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658560:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226666.807607:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720658496:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226666.807627:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658624:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226666.819939:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226666.819951:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226666.819963:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720658688:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226666.863995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226667.378706:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226667.887987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226668.400050:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226668.912012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226669.424063:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226669.936088:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226670.450935:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226670.959961:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226671.472056:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226671.920059:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226671.920068:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226671.920072:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226671.920083:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2265 00000100:00080000:2.0:1761226671.920087:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226671.920094:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226671.920105:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226671.920135:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c9f23bda x1846777720658944/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226671.920173:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226671.920177:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226671.920211:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720658944:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226671.920574:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720658752:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226671.920627:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658880:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:2.0:1761226671.920651:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761226671.920656:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658816:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226671.927636:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226671.927648:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226671.927658:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720658944:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226671.928141:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658816:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226671.928153:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720658752:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226671.928162:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720658880:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226671.983988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226672.496069:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226673.008952:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226673.520191:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226674.032583:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226674.545517:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226675.056399:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226675.569084:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226676.080000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226676.592445:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226677.040300:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720659008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226677.040543:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226677.040550:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720659072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226677.041881:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720659136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226677.044540:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226677.044551:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226677.044555:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226677.044565:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2270 00000100:00080000:0.0:1761226677.044568:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226677.044573:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226677.044584:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226677.044611:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a99a8bd5 x1846777720659200/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226677.044678:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720659200:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761226677.047108:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226677.047141:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761226677.054400:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226677.054411:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226677.054421:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720659200:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226677.054520:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720659136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226677.054533:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720659072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226677.054922:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720659008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226677.103983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226677.617599:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226678.128000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226678.642091:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226679.152022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226679.664274:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226680.175971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226680.693441:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226681.199969:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226681.715381:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226682.160116:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226682.160141:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226682.160146:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226682.160157:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2275 00000100:00080000:1.0:1761226682.160160:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226682.160166:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226682.160176:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226682.160193:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017ee9697 x1846777720659456/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226682.160234:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226682.160238:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226682.160328:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720659456:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226682.162013:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720659264:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226682.163019:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720659392:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226682.163073:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226682.163079:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720659328:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226682.168141:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226682.168156:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226682.168166:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720659456:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226682.171949:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720659264:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226682.174753:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720659328:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226682.174774:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720659392:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226682.223984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226682.740735:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226683.247982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226683.761651:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226684.272445:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226684.784236:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226685.296231:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226685.812683:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226686.319991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226686.833645:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226687.280101:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226687.280110:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226687.280114:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226687.280124:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2280 00000100:00080000:3.0:1761226687.280128:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226687.280133:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226687.280144:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226687.280159:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009d0a2521 x1846777720659712/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226687.280185:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226687.280189:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226687.280213:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720659584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226687.280319:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761226687.280348:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720659520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226687.280378:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720659648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226687.280934:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720659712:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226687.285492:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720659520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226687.286776:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720659648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226687.286792:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720659584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226687.288810:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226687.288819:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226687.288828:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720659712:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226687.344710:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226687.857354:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226688.369025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226688.882256:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226689.391988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226689.905141:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226690.415984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226690.928067:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226691.439979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226691.952063:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226692.400133:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720659776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226692.401779:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226692.401786:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226692.401790:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226692.401799:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2285 00000100:00080000:0.0:1761226692.401803:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226692.401809:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226692.401819:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226692.401835:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720659968/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226692.401897:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720659968:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226692.402068:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720659840:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226692.402106:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226692.402112:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720659904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226692.403414:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226692.403422:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226692.404086:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720659904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226692.404124:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720659840:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226692.404235:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226692.404246:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226692.404251:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720659968:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226692.404720:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720659776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226692.467426:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226692.976259:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226693.488023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226694.001759:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226694.512032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226695.024011:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226695.537668:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226696.048263:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226696.561713:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226697.072196:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226697.520184:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226697.520193:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226697.520196:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226697.520206:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2290 00000100:00080000:1.0:1761226697.520210:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226697.520216:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226697.520226:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226697.520240:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720660224/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226697.520309:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720660224:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00080000:1.0:1761226697.520412:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226697.520416:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226697.520592:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720660032:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226697.520622:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720660160:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226697.520645:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226697.520649:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720660096:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226697.528476:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720660160:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226697.528525:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720660096:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226697.528540:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720660032:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:2.0:1761226697.531191:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226697.531201:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226697.531210:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720660224:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:0.0:1761226697.584479:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226698.095992:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226698.608019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226699.120626:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226699.632031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226700.146968:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226700.655995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226701.168165:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226701.680040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226702.192024:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226702.641284:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226702.641292:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226702.641296:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226702.641306:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2295 00000100:00080000:0.0:1761226702.641309:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226702.641316:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226702.641327:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226702.641342:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d88fcfe6 x1846777720660480/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226702.641390:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226702.641394:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226702.641425:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720660352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226702.641516:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761226702.641522:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720660288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226702.641548:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720660416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226702.641676:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720660480:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226702.647241:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720660352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226702.647552:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720660288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226702.647565:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720660416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226702.655200:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226702.655211:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226702.655221:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720660480:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226702.705158:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226703.219538:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226703.727993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226704.240202:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226704.751951:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226705.269978:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226705.779016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226706.288432:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226706.799959:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226707.312076:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226707.760102:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226707.760110:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226707.760114:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226707.760129:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2300 00000100:00080000:0.0:1761226707.760132:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226707.760139:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226707.760148:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226707.760181:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7988e14 x1846777720660736/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226707.760205:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226707.760209:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226707.760360:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720660608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226707.760449:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226707.760457:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720660544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226707.760489:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720660672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226707.760662:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720660736:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226707.766202:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720660544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226707.766251:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720660672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226707.766262:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720660608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226707.768580:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226707.768592:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226707.768601:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720660736:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226707.824261:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226708.336657:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226708.848003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226709.361164:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226709.872106:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226710.385240:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226710.896513:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226711.408201:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226711.920188:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226712.432087:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226712.880610:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226712.880618:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226712.880622:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226712.880633:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2306 00000100:00080000:3.0:1761226712.880637:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1761226712.880638:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720660800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226712.880643:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226712.880653:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226712.880668:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092860b64 x1846777720660992/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226712.880685:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226712.880689:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226712.880724:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720660864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226712.880737:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720660928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226712.880826:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720660992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761226712.892825:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226712.892835:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226712.892840:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720660992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226712.892994:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720660864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226712.893382:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720660800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226712.893403:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720660928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226712.945748:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226713.456742:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226713.967998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226714.483072:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226714.992012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226715.504097:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226716.016017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226716.528275:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226717.040023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226717.552067:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226718.002663:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226718.002672:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226718.002676:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226718.002687:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2311 00000100:00080000:3.0:1761226718.002691:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226718.002697:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226718.002707:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226718.002722:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd946368 x1846777720661248/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226718.002745:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226718.002748:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226718.003064:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720661248:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226718.003239:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720661056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226718.003299:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720661184:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226718.003323:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226718.003328:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720661120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226718.016506:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226718.016517:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226718.016527:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720661248:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226718.016630:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720661120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226718.016641:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720661056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226718.016651:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720661184:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226718.065584:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226718.576177:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226719.088023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226719.600253:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226720.111986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226720.627321:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226721.136050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226721.648532:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226722.160013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226722.675644:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226723.120065:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226723.120073:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226723.120077:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226723.120087:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2316 00000100:00080000:3.0:1761226723.120091:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226723.120098:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226723.120108:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226723.120141:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a6d0bfc0 x1846777720661504/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761226723.120146:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720661312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226723.120170:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226723.120174:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226723.120199:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720661376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226723.120252:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720661440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226723.120434:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720661504:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226723.130201:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720661312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226723.130404:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720661440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226723.130482:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720661376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:0.0:1761226723.130979:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:0.0:1761226723.138439:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226723.138451:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226723.138463:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720661504:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226723.184060:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226723.696933:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226724.207988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226724.720808:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226725.231973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226725.746339:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226726.256001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226726.769245:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226727.279977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226727.792063:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226728.246126:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226728.246134:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226728.246138:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226728.246150:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2321 00000100:00080000:1.0:1761226728.246154:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226728.246161:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226728.246173:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226728.246190:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dc5f36b1 x1846777720661760/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226728.246221:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226728.246226:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226728.246253:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720661632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226728.246353:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226728.246359:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720661568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226728.246382:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720661696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226728.246766:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720661760:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226728.250609:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720661568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226728.261840:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720661632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226728.263155:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720661696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226728.266479:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226728.266490:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226728.266499:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720661760:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226728.304053:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226728.816062:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226729.327993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226729.840114:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226730.352012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226730.864044:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226731.375973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226731.891352:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226732.403896:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226732.915815:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226733.360089:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226733.360097:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226733.360101:0:36:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226733.360111:0:36:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2326 00000100:00080000:2.0:1761226733.360114:0:36:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226733.360121:0:36:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226733.360131:0:36:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226733.360146:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046bdcfa1 x1846777720662016/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226733.360196:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226733.360200:0:36:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226733.360226:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720661888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226733.360247:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720662016:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:3.0:1761226733.360402:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720661824:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226733.361016:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720661952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226733.366985:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720661824:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00080000:0.0:1761226733.368008:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226733.368016:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226733.368022:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720662016:192.168.202.114@tcp:8:kworker/u8:1.0 00000100:00100000:2.0:1761226733.368080:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720661888:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:3.0:1761226733.368126:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720661952:192.168.202.114@tcp:400:kworker/u8:1.0 00000100:00100000:0.0:1761226733.424027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226733.937043:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226734.447972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226734.960581:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226735.472050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226735.984158:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226736.497518:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226737.012844:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226737.520010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226738.032368:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226738.480126:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226738.480134:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226738.480137:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226738.480149:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2331 00000100:00080000:0.0:1761226738.480152:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226738.480159:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226738.480182:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226738.480198:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720662272/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226738.480219:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226738.480223:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226738.480250:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226738.480343:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226738.480349:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720662080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226738.480369:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226738.480876:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720662272:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226738.484233:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720662080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226738.484255:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226738.484264:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226738.484897:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226738.484905:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226738.484911:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720662272:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226738.544896:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226739.057236:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226739.567955:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226740.080120:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226740.592101:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226741.104358:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226741.615997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226742.128066:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226742.640017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226743.152052:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226743.600805:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720662336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226743.603461:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226743.605030:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226743.605048:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662464:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226743.605381:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720662336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226743.605474:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226743.605481:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226743.605484:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226743.605490:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2336 00000100:00080000:1.0:1761226743.605492:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226743.605498:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226743.605509:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226743.605531:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720662528/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226743.605547:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226743.605551:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226743.605685:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720662528:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226743.611714:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662464:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226743.611732:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226743.616467:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226743.616480:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226743.616490:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720662528:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226743.667021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226744.176070:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226744.688211:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226745.201900:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226745.716007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226746.224138:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226746.738126:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226747.248032:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226747.760007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226748.273361:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226748.720052:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720662592:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226748.723221:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226748.723227:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226748.723231:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226748.723241:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2341 00000100:00080000:1.0:1761226748.723244:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226748.723249:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226748.723258:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226748.723271:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720662784/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226748.723293:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226748.723297:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226748.723332:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226748.723407:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226748.723412:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662656:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226748.725426:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720662784:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226748.726721:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720662592:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226748.732544:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226748.732602:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720662656:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226748.734459:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226748.734468:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226748.734476:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720662784:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226748.783963:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226749.296066:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226749.809689:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226750.320095:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226750.832489:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226751.344187:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226751.856036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226752.368734:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226752.880201:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226753.392828:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226753.841254:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226753.841262:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226753.841266:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226753.841276:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2346 00000100:00080000:3.0:1761226753.841280:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226753.841286:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226753.841313:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226753.841328:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000341ad5cd x1846777720663040/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226753.841354:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226753.841357:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226753.841439:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720662848:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226753.841555:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720662976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226753.841589:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226753.841594:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720662912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226753.841604:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720663040:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761226753.847055:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226753.847067:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226753.847077:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720663040:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226753.847616:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720662912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226753.847631:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720662848:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226753.847642:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720662976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226753.904936:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226754.416946:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226754.927987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226755.440037:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226755.952005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226756.466055:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226756.977088:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226757.488718:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226758.000713:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226758.512071:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226758.961460:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720663104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226758.962274:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226758.962283:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720663168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226758.962724:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226758.962730:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226758.962733:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226758.962738:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2352 00000100:00080000:1.0:1761226758.962741:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226758.962746:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226758.962755:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226758.962767:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dc5f36b1 x1846777720663296/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761226758.962824:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720663296:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761226758.963772:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226758.963779:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226758.963802:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226758.963815:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720663232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226758.966565:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226758.966577:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226758.966586:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720663296:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226758.967754:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720663232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226758.967801:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720663168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226758.967812:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720663104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226759.026201:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226759.536256:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226760.048029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226760.560136:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226761.071997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226761.584091:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226762.095977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226762.608092:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226763.119997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226763.632182:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226764.080075:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226764.080084:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226764.080089:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226764.080102:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2357 00000100:00080000:1.0:1761226764.080106:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761226764.080108:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720663360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226764.080113:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226764.080125:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226764.080142:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720663552/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226764.080203:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720663488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226764.080223:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720663552:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226764.080241:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226764.080250:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720663424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226764.080289:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226764.080293:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226764.088582:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720663424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226764.088601:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720663360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226764.088611:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720663488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226764.088805:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226764.088810:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226764.088814:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720663552:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226764.144119:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226764.656277:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226765.168132:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226765.684932:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226766.191962:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226766.704335:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226767.216076:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226767.730819:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226768.240902:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226768.752162:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226769.200078:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720663616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226769.200513:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226769.200520:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720663680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226769.202032:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226769.202039:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226769.202043:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226769.202050:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2362 00000100:00080000:0.0:1761226769.202053:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226769.202059:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226769.202068:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226769.202084:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720663808/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761226769.202156:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720663808:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226769.202257:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226769.202269:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720663744:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226769.202571:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226769.202575:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226769.208959:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720663680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226769.209005:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720663616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226769.210398:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720663744:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226769.211319:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226769.211327:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226769.211333:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720663808:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226769.264026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226769.777132:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226770.287970:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226770.801161:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226771.311969:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226771.824030:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226772.335992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226772.848810:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226773.359977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226773.874600:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226774.320093:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226774.320103:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226774.320107:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226774.320119:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2367 00000100:00080000:0.0:1761226774.320122:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226774.320129:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226774.320139:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226774.320156:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b0db11ae x1846777720664064/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226774.320184:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226774.320187:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226774.320214:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720663936:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226774.320232:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720664064:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226774.320322:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761226774.320329:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720663872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226774.320360:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720664000:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226774.330224:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720663872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226774.331283:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226774.331301:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226774.331308:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720664064:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226774.331356:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720664000:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226774.331369:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720663936:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226774.383953:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226774.897071:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226775.408714:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226775.920374:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226776.432971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226776.944377:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226777.455977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226777.968499:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226778.480022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226778.993491:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226779.440092:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226779.440100:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226779.440105:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226779.440117:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2372 00000100:00080000:0.0:1761226779.440121:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761226779.440126:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720664192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226779.440131:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226779.440144:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226779.440163:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e956e16 x1846777720664320/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226779.440184:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226779.440188:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226779.440227:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720664128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226779.440335:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720664256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226779.440465:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720664320:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226779.446517:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720664192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226779.446560:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720664128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226779.446570:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720664256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226779.447755:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226779.447762:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226779.447768:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720664320:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226779.504008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226780.016082:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226780.528153:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226781.040101:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226781.552035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226782.064474:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226782.576010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226783.088299:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226783.600013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226784.112102:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226784.561393:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226784.561403:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226784.561408:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226784.561421:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2377 00000100:00080000:1.0:1761226784.561424:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226784.561432:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226784.561442:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226784.561468:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ef613dc x1846777720664576/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226784.561491:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226784.561495:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226784.561538:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720664448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226784.561539:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720664576:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226784.561642:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720664384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226784.564233:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720664512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226784.569403:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1761226784.569407:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720664384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226784.569413:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226784.569418:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720664576:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226784.569423:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720664512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226784.569493:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720664448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226784.624012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226785.136024:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226785.648028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226786.161427:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226786.678848:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226787.184625:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226787.696272:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226788.208121:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226788.719979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226789.234274:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226789.680915:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720664640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226789.682416:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720664704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226789.684623:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226789.684631:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226789.684634:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226789.684644:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2382 00000100:00100000:2.0:1761226789.684645:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720664768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226789.684648:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226789.684693:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226789.684704:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226789.684721:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abb9f4e4 x1846777720664832/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226789.684757:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226789.684760:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226789.684765:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720664832:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226789.686554:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720664640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226789.695147:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720664768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226789.695260:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720664704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226789.708354:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226789.708368:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226789.708378:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720664832:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226789.744003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226790.256289:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226790.768323:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226791.280017:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226791.792004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226792.304082:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226792.818755:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226793.328064:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226793.840483:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226794.352124:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226794.800478:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226794.800487:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226794.800491:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226794.800501:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2387 00000100:00080000:0.0:1761226794.800505:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226794.800520:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226794.800530:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226794.800545:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000de75897f x1846777720665088/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226794.800569:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226794.800572:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226794.800625:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720665088:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226794.801117:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720664896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226794.805784:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720665024:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226794.806809:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761226794.806823:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720664960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226794.814522:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720664896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226794.814547:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720665024:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226794.816887:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720664960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226794.818395:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226794.818406:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226794.818466:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720665088:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226794.864091:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226795.376046:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226795.887967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226796.400358:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226796.911986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226797.428094:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226797.936100:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226798.451350:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226798.959994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226799.472513:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226799.920397:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720665152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226799.920895:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226799.920904:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720665216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226799.921189:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226799.921195:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226799.921198:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226799.921205:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2393 00000100:00080000:2.0:1761226799.921208:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226799.921214:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226799.921224:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226799.921239:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007221726f x1846777720665344/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226799.921292:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720665344:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226799.921674:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226799.921688:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720665280:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226799.922600:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720665152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226799.926575:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226799.926585:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226799.926923:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720665216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226799.926986:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226799.926992:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226799.926997:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720665344:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226799.927692:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720665280:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226799.984105:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226800.497215:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226801.008086:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226801.520002:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226802.032023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226802.547640:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226803.056016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226803.568026:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226804.080058:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226804.592035:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226805.040192:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720665408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226805.041413:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226805.041420:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226805.041424:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761226805.041428:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720665536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226805.041433:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2398 00000100:00080000:1.0:1761226805.041438:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226805.041444:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226805.041453:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226805.041470:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720665600/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226805.041495:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226805.041502:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720665472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226805.041541:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720665600:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761226805.042755:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226805.042764:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226805.049326:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720665536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226805.049347:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720665408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226805.049401:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720665472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226805.050397:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226805.050403:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226805.050410:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720665600:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226805.103987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226805.616095:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226806.127984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226806.642804:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226807.152011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226807.664142:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226808.175985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226808.688173:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226809.200037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226809.712150:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226810.163299:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226810.163307:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226810.163312:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226810.163322:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2403 00000100:00080000:3.0:1761226810.163326:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226810.163333:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226810.163343:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226810.163376:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002b7677b0 x1846777720665856/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226810.163407:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226810.163411:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226810.163437:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720665728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226810.163563:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761226810.163570:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720665664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226810.163593:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720665792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226810.163760:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720665856:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226810.172091:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720665664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226810.172118:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720665728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226810.172817:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226810.172823:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226810.172828:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720665856:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226810.172941:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720665792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226810.224056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226810.736261:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226811.250949:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226811.760150:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226812.272045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226812.785092:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226813.295979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226813.808119:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226814.319987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226814.832063:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226815.280046:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226815.280056:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226815.280060:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226815.280072:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2408 00000100:00080000:3.0:1761226815.280075:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226815.280082:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226815.280098:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226815.280119:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000088958979 x1846777720666112/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226815.280165:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226815.280169:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226815.280279:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720666112:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226815.280342:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720665920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226815.280383:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720666048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226815.280416:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226815.280424:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720665984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226815.284399:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226815.284411:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226815.284420:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720666112:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226815.284424:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720665984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226815.284441:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720666048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226815.284720:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720665920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226815.343983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226815.856111:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226816.368223:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226816.881426:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226817.395055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226817.910730:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226818.415991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226818.927997:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226819.441878:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226819.952006:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226820.400122:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226820.400205:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226820.400209:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226820.400221:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2413 00000100:00080000:3.0:1761226820.400225:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226820.400231:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226820.400241:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226820.400258:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000074fca4bf x1846777720666368/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226820.400303:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226820.400307:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226820.400332:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720666240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226820.400338:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720666368:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226820.400460:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720666176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226820.400574:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720666304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226820.409740:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226820.409753:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226820.409764:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720666368:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226820.414232:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720666240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226820.414286:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720666176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226820.414875:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720666304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226820.463979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226820.977375:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226821.490946:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226822.000031:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226822.514027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226823.025703:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226823.536035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226824.050148:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226824.560022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226825.073167:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226825.520055:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226825.520063:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00100000:2.0:1761226825.520067:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720666432:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226825.520067:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226825.520074:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2418 00000100:00080000:3.0:1761226825.520077:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226825.520083:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226825.520092:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226825.520105:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003d27cd56 x1846777720666624/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226825.520130:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226825.520133:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226825.520152:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720666496:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226825.520167:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720666560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226825.520255:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720666624:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226825.523942:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720666432:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226825.529691:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226825.529703:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226825.529719:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720666624:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226825.530267:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720666496:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226825.530281:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720666560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226825.585656:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226826.098033:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226826.608075:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226827.123300:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226827.632480:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226828.144073:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226828.656010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226829.168387:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226829.679979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226830.192245:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226830.640110:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720666688:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226830.641571:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720666752:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226830.643337:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720666816:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226830.647016:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720666688:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226830.648605:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226830.648639:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226830.648643:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226830.648656:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2423 00000100:00080000:3.0:1761226830.648659:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226830.648666:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226830.648691:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226830.648720:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003d27cd56 x1846777720666880/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226830.648753:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226830.648756:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226830.649612:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720666880:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226830.650175:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720666752:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226830.658284:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720666816:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226830.662730:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226830.662742:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226830.662752:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720666880:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226830.704151:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226831.216016:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226831.728042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226832.240075:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226832.751944:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226833.264247:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226833.778051:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226834.288054:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226834.799925:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226835.312655:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226835.760049:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226835.760058:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226835.760061:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226835.760071:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2428 00000100:00080000:1.0:1761226835.760075:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226835.760082:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226835.760093:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226835.760109:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720667136/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226835.760151:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226835.760155:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226835.760185:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226835.760274:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761226835.760281:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720666944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226835.760302:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226835.760404:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720667136:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226835.765761:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720666944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226835.769742:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226835.769764:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226835.769775:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720667136:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226835.770101:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226835.770123:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226835.827596:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226836.336055:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226836.849740:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226837.360217:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226837.872136:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226838.384093:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226838.901105:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226839.409172:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226839.920038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226840.432283:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761226840.880225:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720667200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226840.880685:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761226840.880694:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226840.881661:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226840.881685:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226840.881689:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226840.881697:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2434 00000100:00080000:0.0:1761226840.881702:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226840.881753:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226840.881767:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226840.881787:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720667392/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761226840.881844:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720667392:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226840.881993:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226840.883412:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226840.883422:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761226840.889126:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226840.889142:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226840.889154:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720667392:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226840.891438:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226840.891459:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720667200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226840.901511:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226840.944674:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226841.457269:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226841.970943:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226842.481132:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226842.992006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226843.506798:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226844.015979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226844.528526:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226845.040042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226845.552706:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226846.002367:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226846.002376:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226846.002380:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226846.002395:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2439 00000100:00080000:1.0:1761226846.002421:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226846.002430:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226846.002443:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226846.002463:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abc685ce x1846777720667648/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226846.002486:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226846.002489:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226846.004031:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720667648:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226846.004677:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720667456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226846.004732:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720667584:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226846.004761:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226846.004768:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720667520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226846.009697:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226846.009711:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226846.009724:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720667648:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226846.009818:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720667456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226846.009833:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720667584:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226846.009864:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720667520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226846.063980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226846.576149:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226847.089731:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226847.600203:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226848.111991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226848.625118:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226849.135983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226849.648139:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226850.160395:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226850.677396:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226851.120102:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226851.120110:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226851.120115:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226851.120128:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2444 00000100:00080000:1.0:1761226851.120133:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226851.120140:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226851.120153:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226851.120171:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000158429d4 x1846777720667904/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226851.120191:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226851.120194:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226851.120306:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720667712:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226851.120384:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720667904:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226851.121965:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667840:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226851.122498:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720667776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226851.133094:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226851.133106:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226851.133118:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720667904:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226851.133267:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720667712:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226851.135288:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720667840:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226851.135386:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720667776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226851.184033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226851.697152:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226852.211179:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226852.720573:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226853.232081:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226853.744741:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226854.256025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226854.768174:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226855.279993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226855.792116:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761226856.240533:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226856.240540:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761226856.240544:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761226856.240556:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2449 00000100:00080000:0.0:1761226856.240559:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761226856.240565:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761226856.240579:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761226856.240597:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fe9fa97 x1846777720668160/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761226856.240621:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761226856.240624:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226856.240656:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720668032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226856.240729:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720668160:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226856.240769:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761226856.240780:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720667968:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226856.240822:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720668096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226856.245149:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720667968:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226856.245199:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720668096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226856.245214:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720668032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226856.245303:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226856.245308:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226856.245313:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720668160:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226856.304017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226856.817389:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226857.328586:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226857.840303:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226858.352027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226858.865407:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226859.375974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226859.888155:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226860.400761:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226860.914440:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226861.360141:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226861.360149:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226861.360153:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226861.360179:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2454 00000100:00080000:3.0:1761226861.360183:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226861.360189:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226861.360200:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226861.360215:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035883416 x1846777720668416/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226861.360238:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226861.360242:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226861.360309:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720668416:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226861.360485:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720668224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226861.360603:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720668352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226861.360631:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226861.360637:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720668288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226861.364674:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720668288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226861.364694:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720668352:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226861.366633:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720668224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226861.366951:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226861.366957:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226861.366962:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720668416:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226861.424044:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226861.939066:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226862.447961:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226862.960496:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226863.473719:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226863.984375:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226864.495987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226865.008186:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226865.519982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226866.032849:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226866.482400:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226866.482410:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226866.482415:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226866.482425:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2459 00000100:00080000:2.0:1761226866.482429:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226866.482435:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226866.482445:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226866.482460:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007221726f x1846777720668672/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226866.482486:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226866.482489:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226866.482517:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720668544:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226866.482540:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720668672:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226866.482627:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720668480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226866.483909:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720668608:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226866.492606:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720668544:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226866.492900:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720668480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226866.492914:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720668608:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226866.493010:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226866.493015:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226866.493019:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720668672:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226866.545772:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226867.056266:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226867.568060:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226868.081956:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226868.592576:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226869.104114:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226869.616384:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226870.129978:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226870.639981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226871.152116:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226871.600081:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226871.600088:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226871.600092:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226871.600101:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2464 00000100:00080000:1.0:1761226871.600105:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226871.600111:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226871.600121:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226871.600136:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720668928/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226871.600164:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226871.600168:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226871.600194:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720668800:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226871.600330:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761226871.600337:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720668736:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226871.600362:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720668864:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226871.600582:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720668928:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226871.612113:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720668736:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226871.612141:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720668864:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226871.612156:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720668800:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761226871.612602:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226871.612608:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226871.612614:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720668928:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226871.663949:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226872.179678:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226872.687979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226873.203711:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226873.714208:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226874.225233:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226874.736757:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226875.250782:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226875.760043:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226876.274286:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226876.720500:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226876.720508:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226876.720512:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226876.720525:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2469 00000100:00080000:1.0:1761226876.720528:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226876.720536:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226876.720549:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226876.720569:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017ee9697 x1846777720669184/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761226876.720634:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720669184:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761226876.720770:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226876.720774:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761226876.721257:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720668992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226876.721296:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720669120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226876.721326:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761226876.721331:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720669056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:3.0:1761226876.726770:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:3.0:1761226876.726800:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761226876.726806:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761226876.726810:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720669184:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226876.728721:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720668992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226876.729775:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720669056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226876.729790:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720669120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226876.784101:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226877.296119:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226877.808710:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226878.320140:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226878.832020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226879.345842:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226879.857575:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226880.368087:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226880.879984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226881.393637:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226881.840356:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226881.840364:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226881.840369:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226881.840380:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2474 00000100:00080000:2.0:1761226881.840384:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226881.840390:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226881.840400:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226881.840416:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fab9894 x1846777720669440/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226881.840443:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226881.840447:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226881.840636:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720669248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226881.840731:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720669376:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226881.840764:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226881.840769:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720669312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226881.842029:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720669440:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226881.849221:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720669312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226881.849241:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720669248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226881.849250:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720669376:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226881.849892:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226881.849899:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226881.849905:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720669440:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226881.903984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226882.416392:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226882.929251:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226883.444158:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226883.954608:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226884.464546:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226884.977435:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226885.491998:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226885.999986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226886.514104:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226886.960390:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720669504:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226886.968118:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226886.968126:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226886.968130:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226886.968141:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2480 00000100:00080000:1.0:1761226886.968144:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226886.968166:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226886.968177:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226886.968194:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b1564d60 x1846777720669696/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761226886.968259:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720669696:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761226886.968328:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720669632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226886.968371:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720669504:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226886.968391:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226886.968395:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720669568:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226886.969504:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226886.969511:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761226886.971543:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226886.971550:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226886.971559:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720669696:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226886.973510:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720669632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226886.973932:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720669568:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226887.024129:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226887.538624:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226888.050217:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226888.560069:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226889.074161:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226889.584202:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226890.097115:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226890.608600:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226891.123178:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226891.632256:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226892.080225:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226892.080233:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226892.080237:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226892.080248:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2485 00000100:00080000:1.0:1761226892.080251:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226892.080256:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226892.080267:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226892.080282:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d0d44bb x1846777720669952/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226892.080307:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226892.080310:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226892.080336:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720669824:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226892.080423:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761226892.080429:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720669760:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226892.080467:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720669888:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226892.081103:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720669952:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761226892.087032:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720669760:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226892.087057:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720669888:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226892.087068:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720669824:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226892.102711:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226892.102723:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226892.102733:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720669952:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226892.143986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226892.656162:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226893.169361:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226893.683620:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226894.192010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226894.704912:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226895.217492:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226895.728088:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226896.240134:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226896.754806:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226897.200128:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226897.200137:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226897.200141:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226897.200154:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2490 00000100:00080000:1.0:1761226897.200158:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761226897.200165:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226897.200177:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226897.200196:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001c661ef2 x1846777720670208/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226897.200221:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226897.200224:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226897.200321:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720670208:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226897.200337:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720670016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226897.200472:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720670144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226897.200519:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761226897.200527:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720670080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226897.209262:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720670144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226897.209297:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720670080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226897.209307:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720670016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226897.210036:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226897.210043:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226897.210050:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720670208:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226897.264018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226897.776040:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226898.288289:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226898.807812:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226899.313317:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226899.824135:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226900.336005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226900.848362:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226901.359983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226901.872503:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226902.321533:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720670272:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226902.323368:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226902.323381:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720670336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226902.324545:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720670272:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226902.325497:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720670336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226902.325668:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226902.325674:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226902.325693:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226902.325699:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2495 00000100:00080000:2.0:1761226902.325701:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226902.325707:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226902.325717:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226902.325732:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084e78951 x1846777720670464/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226902.325754:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226902.325757:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226902.325877:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720670400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226902.326007:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720670464:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226902.328302:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720670400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226902.330029:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226902.330039:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226902.330062:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720670464:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226902.383988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226902.896248:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226903.407995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226903.920362:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226904.432133:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226904.944952:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226905.458011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226905.968025:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226906.480196:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226906.994151:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226907.440148:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226907.440156:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226907.440161:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226907.440171:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2500 00000100:00080000:2.0:1761226907.440174:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226907.440181:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226907.440193:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226907.440209:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec5c4aea x1846777720670720/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226907.440234:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226907.440254:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226907.440330:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720670528:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226907.440412:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720670656:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226907.440441:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226907.440448:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720670592:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226907.440690:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720670720:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226907.442444:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720670528:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226907.445177:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720670592:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226907.445193:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720670656:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226907.447215:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226907.447225:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226907.447234:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720670720:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226907.504704:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226908.016379:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226908.528050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226909.042978:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226909.552440:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226910.064369:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226910.575983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226911.088128:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226911.600015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226912.112018:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226912.560077:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226912.560085:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226912.560089:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226912.560100:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2505 00000100:00080000:2.0:1761226912.560103:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226912.560109:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226912.560118:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226912.560134:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000064ce9b52 x1846777720670976/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761226912.560151:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720670784:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226912.560161:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226912.560165:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226912.560205:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720670848:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226912.560256:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720670912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226912.560720:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720670976:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226912.565838:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720670848:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226912.566182:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720670784:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226912.566197:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720670912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226912.567407:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226912.567415:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226912.567422:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720670976:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226912.623987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226913.138106:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226913.648134:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226914.160085:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226914.672149:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761226915.184007:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226915.696436:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226916.208161:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226916.719990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226917.232280:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226917.680103:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720671040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226917.682339:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226917.682347:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226917.682351:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226917.682362:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2510 00000100:00080000:2.0:1761226917.682366:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226917.682372:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226917.682382:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226917.682398:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720671232/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226917.682419:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226917.682423:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226917.682448:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720671168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226917.682537:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720671040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226917.682557:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761226917.682562:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720671104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226917.682707:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720671232:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761226917.691025:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226917.691035:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226917.691045:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720671232:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226917.693207:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720671104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226917.693226:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720671168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226917.744945:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226918.262144:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226918.767995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226919.281760:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226919.791989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226920.305140:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226920.816153:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226921.329644:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226921.840797:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226922.353057:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226922.800052:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226922.800060:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226922.800064:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226922.800075:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2515 00000100:00100000:3.0:1761226922.800077:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720671296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226922.800080:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226922.800090:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226922.800101:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226922.800118:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b3214ada x1846777720671488/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226922.800137:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226922.800140:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226922.800174:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226922.800229:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226922.800236:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226922.800304:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720671488:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226922.808118:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720671296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226922.809666:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226922.809686:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226922.809729:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226922.809750:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226922.809755:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720671488:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226922.863972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226923.377527:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226923.887993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226924.401456:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761226924.912040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226925.425000:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226925.936392:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226926.450282:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226926.963663:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226927.473103:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226927.922656:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720671552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226927.924350:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226927.924358:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226927.924363:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226927.924373:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2521 00000100:00080000:2.0:1761226927.924377:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226927.924385:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226927.924416:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226927.924438:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720671744/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761226927.924506:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720671744:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226927.924988:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226927.925095:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226927.925101:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226927.925513:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226927.925519:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226927.943942:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720671552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226927.944256:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226927.944271:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226927.945046:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226927.945053:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226927.945059:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720671744:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761226927.984049:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226928.497341:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226929.008018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226929.520098:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226930.032027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226930.544217:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226931.056027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226931.568650:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226932.082390:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226932.592066:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226933.041901:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226933.041909:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226933.041914:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226933.041945:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2526 00000100:00080000:3.0:1761226933.041949:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761226933.041956:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226933.041969:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226933.041988:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abb9f4e4 x1846777720672000/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761226933.042070:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720672000:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761226933.042798:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226933.042804:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226933.042839:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720671808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226933.043190:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671936:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226933.043218:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226933.043224:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226933.060346:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761226933.060356:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226933.060359:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226933.060367:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720672000:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226933.060376:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720671808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226933.060390:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720671936:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226933.104037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226933.618606:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226934.127984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226934.640336:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226935.152559:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226935.664870:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226936.176015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226936.688069:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226937.200026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226937.712257:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226938.160165:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720672064:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226938.162254:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226938.162262:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226938.162267:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226938.162277:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2531 00000100:00080000:2.0:1761226938.162281:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761226938.162288:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226938.162301:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226938.162317:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f5134a53 x1846777720672256/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226938.162345:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226938.162350:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226938.162391:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226938.162482:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761226938.162497:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226938.162929:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720672256:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226938.165143:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720672064:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226938.169806:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226938.169820:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226938.169833:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720672256:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226938.169890:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226938.169930:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226938.225726:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226938.744654:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226939.248184:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226939.761449:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226940.272327:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226940.785264:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226941.296011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226941.808374:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226942.319989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226942.832108:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226943.280043:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226943.280052:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226943.280056:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761226943.280061:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720672320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226943.280062:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2536 00000100:00080000:2.0:1761226943.280066:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1761226943.280073:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226943.280084:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226943.280098:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec5c4aea x1846777720672512/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226943.280122:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226943.280125:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226943.280155:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720672384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226943.280156:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720672512:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226943.281547:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226943.286251:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226943.286263:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226943.286273:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720672512:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226943.286440:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720672384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226943.286535:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720672320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226943.286549:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226943.344150:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226943.857868:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226944.367974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226944.887937:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226945.391974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226945.904080:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226946.417659:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226946.928059:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226947.439983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226947.952232:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226948.400084:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226948.400091:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226948.400095:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226948.400105:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2541 00000100:00080000:3.0:1761226948.400109:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1761226948.400114:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226948.400124:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226948.400137:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b01aecf x1846777720672768/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226948.400160:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226948.400162:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226948.400204:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761226948.400213:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720672768:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226948.400298:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720672576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226948.400569:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720672704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226948.402918:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761226948.402927:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761226948.402930:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226948.402938:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720672768:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226948.403039:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720672576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761226948.403050:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720672704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226948.467001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226948.976187:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226949.488015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226950.000142:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226950.512018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226951.024065:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226951.536447:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226952.049889:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761226952.560014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226953.073071:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226953.520052:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226953.520060:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226953.520063:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226953.520074:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2546 00000100:00080000:3.0:1761226953.520079:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 16s 00000100:00080000:3.0:1761226953.520086:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:2.0:1761226953.520096:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720672896:192.168.202.114@tcp:400:kworker/u8:0.0 00000008:00100000:3.0:1761226953.520096:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226953.520113:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720673024/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226953.520136:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226953.520140:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226953.520175:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720672832:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226953.520186:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720673024:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226953.520213:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226953.522614:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720672896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226953.530444:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720672832:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761226953.531878:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761226953.531888:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761226953.531896:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720673024:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226953.531972:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720672960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226953.584667:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226954.096386:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226954.609117:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226955.120109:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226955.632002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226956.144125:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226956.656014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226957.170013:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226957.680137:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226958.195422:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226958.640087:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226958.640095:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226958.640100:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226958.640112:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2551 00000100:00080000:3.0:1761226958.640117:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226958.640123:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226958.640134:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226958.640156:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e86c3a42 x1846777720673280/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226958.640180:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226958.640184:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226958.640227:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226958.640366:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761226958.640373:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720673088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226958.640398:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226958.640531:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720673280:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761226958.647682:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1761226958.647688:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720673088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226958.647692:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226958.647700:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720673280:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226958.647712:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226958.647729:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226958.703992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226959.218887:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226959.727999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226960.240497:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226960.752204:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226961.267103:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226961.776037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226962.288146:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226962.801076:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226963.312123:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226963.760126:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226963.760134:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226963.760138:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226963.760150:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2556 00000100:00080000:3.0:1761226963.760153:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226963.760160:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226963.760172:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226963.760187:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd946368 x1846777720673536/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226963.760202:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226963.760214:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226963.760287:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720673344:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226963.760394:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226963.760425:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761226963.760430:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673408:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226963.760974:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720673536:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761226963.765713:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720673344:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226963.766715:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226963.766931:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673408:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226963.776932:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226963.776946:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226963.776958:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720673536:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226963.823973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226964.336461:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226964.848004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226965.361616:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226965.872591:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226966.384521:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226966.896033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226967.411026:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226967.921848:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226968.433388:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226968.880421:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720673600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226968.881201:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226968.881207:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226968.881210:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761226968.881598:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226968.881659:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226968.881666:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226968.881759:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2562 00000100:00080000:2.0:1761226968.881763:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761226968.881770:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226968.881784:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226968.881807:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072dbf25b x1846777720673792/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761226968.881890:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720673792:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761226968.882919:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226968.882927:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761226968.883762:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720673600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226968.890178:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226968.890205:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720673664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226968.893667:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226968.893682:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226968.893694:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720673792:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226968.948590:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226969.458360:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226969.968040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226970.480226:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226970.993426:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226971.504104:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226972.015978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226972.528782:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226973.040146:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226973.554251:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761226974.000155:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226974.000164:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761226974.000169:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761226974.000183:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2567 00000100:00080000:1.0:1761226974.000187:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761226974.000194:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761226974.000223:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761226974.000243:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057d87282 x1846777720674048/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761226974.000270:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761226974.000274:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761226974.000306:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720673920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226974.000516:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761226974.000523:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720673856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761226974.000552:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720673984:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761226974.000792:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720674048:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761226974.004435:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720673856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226974.006387:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720673984:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226974.006429:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720673920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761226974.009789:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226974.009802:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226974.009814:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720674048:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226974.063985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226974.576373:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226975.089705:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226975.601281:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226976.111984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226976.625692:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226977.137362:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226977.652253:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226978.160037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226978.672089:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226979.122227:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226979.122235:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226979.122257:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226979.122270:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2572 00000100:00080000:3.0:1761226979.122274:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226979.122280:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226979.122292:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226979.122311:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009d0a2521 x1846777720674304/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226979.122342:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226979.122346:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226979.122370:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720674176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226979.122474:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761226979.122481:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720674112:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226979.122505:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720674240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761226979.123346:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720674304:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226979.129080:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720674112:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761226979.129322:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226979.129328:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226979.129333:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720674304:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761226979.129407:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720674240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226979.129420:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720674176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761226979.184287:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226979.696105:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226980.208641:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226980.720117:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226981.232774:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226981.746055:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226982.256090:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226982.772191:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226983.279979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226983.792188:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226984.240263:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226984.240272:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226984.240275:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226984.240285:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2577 00000100:00100000:2.0:1761226984.240287:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720674432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761226984.240289:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226984.240294:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226984.240303:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226984.240319:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009d0a2521 x1846777720674560/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226984.240344:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226984.240347:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226984.240381:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720674368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226984.240409:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720674496:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226984.241934:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720674560:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761226984.244375:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720674432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226984.245692:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720674368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226984.245712:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720674496:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226984.253826:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226984.253838:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226984.253848:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720674560:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226984.306186:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226984.816287:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226985.328283:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226985.840079:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226986.352017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226986.864256:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226987.376204:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761226987.893630:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226988.400683:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226988.914246:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761226989.360839:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226989.360847:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761226989.360851:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761226989.360884:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2582 00000100:00080000:2.0:1761226989.360887:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761226989.360893:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761226989.360903:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761226989.360918:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000790fc921 x1846777720674816/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761226989.360953:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761226989.360956:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226989.361041:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720674624:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226989.361131:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720674752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226989.361160:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761226989.361167:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720674688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226989.361273:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720674816:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226989.365679:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720674624:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226989.365705:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720674752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226989.365735:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720674688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761226989.365812:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226989.365818:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226989.365823:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720674816:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226989.426988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226989.936298:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226990.450922:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226990.960026:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226991.475235:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226991.984116:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226992.495981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226993.010831:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226993.519985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226994.035985:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226994.480940:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226994.480947:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226994.480951:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226994.480962:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2587 00000100:00080000:3.0:1761226994.480965:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226994.480971:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226994.480981:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226994.481008:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f9712cd6 x1846777720675072/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226994.481034:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226994.481037:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226994.481058:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720674880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226994.481138:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720675008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226994.481164:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761226994.481168:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720674944:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226994.481246:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720675072:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761226994.490587:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761226994.490598:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761226994.490607:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720675072:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226994.491129:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720675008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226994.491151:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720674944:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226994.491160:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720674880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226994.543996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226995.059113:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226995.568269:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226996.080303:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226996.591925:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226997.104414:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226997.616001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761226998.128719:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761226998.640069:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761226999.156250:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761226999.600232:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226999.600241:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761226999.600246:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761226999.600260:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2592 00000100:00080000:3.0:1761226999.600265:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761226999.600290:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761226999.600304:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761226999.600324:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c2618360 x1846777720675328/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761226999.600351:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761226999.600356:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761226999.600416:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720675200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761226999.600586:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720675328:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226999.602071:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761226999.602090:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720675136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226999.602734:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720675264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761226999.609840:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761226999.609891:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761226999.609902:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720675328:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761226999.609985:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720675136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226999.609999:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720675264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226999.610010:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720675200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761226999.663988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227000.176225:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227000.688102:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227001.201566:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227001.715093:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227002.224163:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227002.736070:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227003.249198:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227003.762538:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227004.272930:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227004.720277:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227004.720285:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227004.720290:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227004.720304:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2597 00000100:00080000:1.0:1761227004.720308:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227004.720315:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227004.720329:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227004.720348:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720675584/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227004.720376:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227004.720399:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227004.720437:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720675456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227004.720551:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761227004.720558:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720675392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227004.720581:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720675520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227004.720888:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720675584:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227004.725978:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720675392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227004.726009:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720675520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227004.726023:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720675456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227004.733697:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227004.733709:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227004.733718:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720675584:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227004.784820:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227005.296126:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227005.808110:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227006.320056:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227006.831983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227007.349704:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227007.856321:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227008.368256:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227008.884756:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227009.392038:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227009.855226:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227009.855234:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227009.855238:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227009.855252:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2602 00000100:00080000:3.0:1761227009.855255:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761227009.855263:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:2.0:1761227009.855268:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720675648:192.168.202.114@tcp:400:kworker/u8:2.0 00000008:00100000:3.0:1761227009.855278:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227009.855299:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a6d0bfc0 x1846777720675840/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227009.855327:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227009.855332:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227009.855372:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720675712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227009.855832:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720675840:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761227009.857298:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720675776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227009.862781:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720675648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227009.862790:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720675712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761227009.865279:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227009.865294:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227009.865306:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720675840:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761227009.870941:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720675776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227009.904001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227010.420098:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227010.928024:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227011.440771:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227011.951980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227012.464068:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227012.978808:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227013.488125:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227013.999999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227014.512497:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227014.962680:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227014.962687:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227014.962691:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227014.962701:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2608 00000100:00080000:0.0:1761227014.962705:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227014.962728:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227014.962738:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227014.962754:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b0db11ae x1846777720676096/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227014.962778:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227014.962782:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227014.962811:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720675968:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227014.962893:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720676096:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227014.962960:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761227014.962973:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720675904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227014.963015:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720676032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227014.968891:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720676032:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227014.968915:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720675968:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761227014.969990:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227014.970003:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227014.970006:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720675904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227014.970035:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720676096:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227015.023966:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227015.536074:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227016.048000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227016.560203:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227017.072031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227017.584131:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227018.098591:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227018.608129:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227019.119969:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227019.632077:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227020.080100:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227020.080109:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227020.080113:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227020.080124:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2613 00000100:00080000:0.0:1761227020.080127:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227020.080134:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227020.080161:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227020.080179:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720676352/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227020.080206:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227020.080209:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227020.080230:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720676224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227020.080318:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761227020.080323:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720676160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227020.080342:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720676288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227020.080628:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720676352:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227020.084538:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720676160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227020.084561:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720676288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227020.084572:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720676224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227020.090607:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227020.090619:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227020.090629:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720676352:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227020.144889:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227020.657724:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227021.167980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227021.679991:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227022.193161:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227022.704782:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227023.217436:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227023.728734:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227024.240663:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227024.752087:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227025.200715:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720676416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227025.201388:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227025.201395:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227025.201399:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1761227025.201401:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720676480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227025.201408:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2618 00000100:00080000:3.0:1761227025.201412:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761227025.201418:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227025.201429:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227025.201455:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008dec2428 x1846777720676608/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227025.201473:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:3.0:1761227025.201474:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227025.201479:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227025.201482:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720676544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227025.201605:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720676608:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761227025.210491:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227025.210503:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227025.210512:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720676608:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227025.210610:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720676544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227025.210621:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720676480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227025.210646:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720676416:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227025.265706:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227025.776132:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227026.288970:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227026.810730:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227027.313702:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227027.824072:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761227028.274899:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720676736:192.168.202.114@tcp:17:kworker/1:2.0 00000100:00100000:1.0:1761227028.274899:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720676672:192.168.202.114@tcp:17:kworker/1:2.0 00000100:00100000:0.0:1761227028.282516:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720676736:192.168.202.114@tcp:17:kworker/1:2.0 00000100:00100000:1.0:1761227028.282516:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720676672:192.168.202.114@tcp:17:kworker/1:2.0 00000100:00100000:3.0:1761227028.336265:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227028.869679:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227029.361604:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227029.877149:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227030.320170:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720676800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227030.321777:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227030.321789:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720676864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761227030.322815:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227030.322821:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227030.322825:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227030.322832:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2623 00000100:00080000:0.0:1761227030.322835:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227030.322866:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227030.322876:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227030.322890:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d991e6ad x1846777720676992/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227030.322950:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720676992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761227030.324304:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227030.324312:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227030.324370:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227030.324384:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720676928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227030.330091:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720676864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:3.0:1761227030.330599:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00100000:0.0:1761227030.349141:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720676928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227030.349175:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720676800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227030.353877:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227030.353888:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227030.353901:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720676992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227030.384554:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227030.896091:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227031.408027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227031.921584:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227032.433648:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227032.944421:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227033.456291:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227033.968162:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227034.480000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227034.992071:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227035.440679:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227035.440686:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227035.440690:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227035.440701:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2628 00000100:00080000:0.0:1761227035.440705:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227035.440711:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227035.440722:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227035.440738:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720677248/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227035.440763:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227035.440783:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227035.441174:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720677056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227035.441175:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720677248:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227035.441277:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227035.442379:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227035.442392:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677120:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227035.450787:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720677056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227035.456656:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227035.456671:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227035.456682:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720677248:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227035.457621:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227035.458075:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677120:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227035.504002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227036.019268:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227036.527952:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227037.040169:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227037.552091:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227038.066432:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227038.576024:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227039.088455:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227039.600100:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227040.112193:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227040.564296:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227040.564304:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227040.564309:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227040.564323:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2633 00000100:00080000:2.0:1761227040.564349:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761227040.564366:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:3.0:1761227040.564380:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720677312:192.168.202.114@tcp:400:kworker/u8:2.0 00000008:00100000:2.0:1761227040.564381:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227040.564404:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003ba27032 x1846777720677504/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761227040.564476:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720677504:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227040.564511:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720677440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227040.564571:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227040.564586:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720677376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761227040.564666:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227040.564677:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761227040.573105:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227040.573117:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227040.573126:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720677504:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761227040.573248:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720677376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227040.573262:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720677312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227040.573272:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720677440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227040.623982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227041.136090:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227041.648490:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227042.160084:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227042.671987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227043.184465:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227043.695981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227044.208088:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227044.720932:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227045.233498:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227045.680118:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227045.680125:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227045.680130:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227045.680140:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2638 00000100:00080000:1.0:1761227045.680143:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227045.680149:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227045.680159:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227045.680173:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720677760/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227045.680200:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227045.680204:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227045.680230:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227045.680325:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761227045.680331:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720677568:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227045.680354:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227045.680939:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720677760:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227045.690941:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720677568:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227045.690972:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227045.692990:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227045.694896:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227045.694907:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227045.694916:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720677760:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227045.745330:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227046.256122:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227046.769406:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227047.280107:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227047.792023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227048.306838:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227048.816009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227049.331726:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227049.840016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227050.356133:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761227050.800147:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720677824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227050.800650:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227050.800658:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677888:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227050.800740:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227050.800746:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227050.800750:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227050.800756:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2643 00000100:00080000:1.0:1761227050.800759:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227050.800767:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227050.800780:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227050.800796:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094e95ca3 x1846777720678016/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227050.800819:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227050.800822:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227050.800843:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227050.800851:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720677952:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227050.800887:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720678016:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227050.815531:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720677952:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227050.815891:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720677888:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227050.815927:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720677824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227050.817907:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227050.817919:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227050.817929:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720678016:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227050.864010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227051.376030:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227051.888011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227052.400704:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227052.912392:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227053.424015:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227053.936026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227054.448046:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227054.960774:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227055.472068:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227055.920138:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720678080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227055.921471:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227055.922559:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720678208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761227055.923261:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227055.923268:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227055.923271:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227055.923277:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2649 00000100:00080000:0.0:1761227055.923281:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227055.923288:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227055.923300:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227055.923320:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003e02fcda x1846777720678272/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227055.923337:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227055.923340:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227055.933281:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720678080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227055.933791:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720678272:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761227055.934196:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227055.934290:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720678208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227055.937311:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227055.937336:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227055.937347:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720678272:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227055.986934:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227056.497968:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227057.008781:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227057.520749:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227058.032370:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227058.545310:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227059.056061:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227059.568143:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227060.079953:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227060.593573:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227061.040148:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227061.040157:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227061.040161:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227061.040172:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2654 00000100:00080000:1.0:1761227061.040175:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227061.040182:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227061.040193:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227061.040210:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abc685ce x1846777720678528/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761227061.040288:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720678528:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:1.0:1761227061.040401:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227061.040405:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227061.040481:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720678336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227061.040514:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678464:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227061.040543:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227061.040570:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227061.044925:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678464:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227061.050338:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227061.050349:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227061.050359:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720678528:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227061.050456:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227061.050466:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720678336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227061.110051:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227061.622022:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227062.128032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227062.640313:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227063.152023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227063.664848:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227064.175971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227064.694373:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227065.200016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227065.712163:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227066.164141:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227066.164150:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227066.164154:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227066.164168:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2659 00000100:00080000:0.0:1761227066.164172:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227066.164179:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227066.164194:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227066.164213:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720678784/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227066.164241:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227066.164245:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227066.164350:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678656:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227066.164439:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720678592:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227066.164478:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720678720:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227066.164724:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720678784:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227066.181630:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678656:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227066.181681:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720678592:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227066.181694:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720678720:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227066.188640:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227066.188652:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227066.188661:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720678784:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227066.224145:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227066.736400:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227067.250504:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227067.764149:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227068.272233:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227068.784268:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227069.296359:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227069.808106:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227070.320731:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227070.832190:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227071.280047:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227071.280055:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227071.280059:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227071.280069:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2664 00000100:00080000:0.0:1761227071.280073:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227071.280080:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227071.280091:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227071.280126:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720679040/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227071.280152:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227071.280155:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227071.280246:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720679040:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227071.280900:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720678848:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227071.280942:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227071.280967:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227071.280973:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227071.288647:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227071.288669:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720678976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227071.297668:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227071.297681:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227071.297690:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720679040:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227071.297902:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720678848:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227071.344530:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227071.856099:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227072.369831:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227072.880085:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227073.392009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227073.908083:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227074.416124:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227074.928180:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227075.440241:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227075.952636:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227076.400491:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227076.400500:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227076.400504:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227076.400516:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2669 00000100:00080000:1.0:1761227076.400521:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227076.400540:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227076.400554:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227076.400575:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a2fc13a x1846777720679296/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227076.400603:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227076.400607:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227076.400643:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227076.400770:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761227076.400778:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720679104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227076.400804:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227076.401117:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720679296:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227076.404132:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227076.404168:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227076.408511:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720679104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761227076.408595:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227076.408668:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227076.408674:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720679296:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227076.464265:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227076.977267:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227077.491138:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227078.000109:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227078.513073:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227079.024792:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227079.536404:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227080.052988:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227080.560804:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227081.073092:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227081.520168:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227081.520182:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227081.520186:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227081.520198:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2674 00000100:00080000:3.0:1761227081.520202:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761227081.520209:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:2.0:1761227081.520219:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720679360:192.168.202.114@tcp:400:kworker/u8:0.0 00000008:00100000:3.0:1761227081.520221:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227081.520236:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e86c3a42 x1846777720679552/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227081.520263:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227081.520267:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227081.520322:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720679424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227081.520378:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720679552:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227081.520424:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720679488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227081.526708:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227081.526723:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227081.526734:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720679552:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227081.528977:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720679360:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227081.528997:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720679488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227081.529172:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720679424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227081.587464:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227082.096368:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227082.608019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227083.120777:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227083.632009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227084.145201:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227084.658455:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227085.169772:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227085.679998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227086.192100:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227086.640167:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227086.640176:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227086.640180:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227086.640191:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2679 00000100:00080000:1.0:1761227086.640195:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227086.640201:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227086.640213:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227086.640228:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001c661ef2 x1846777720679808/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227086.640258:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227086.640261:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227086.640315:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720679808:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227086.640425:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720679616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227086.640464:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679744:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227086.640498:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227086.640505:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227086.643801:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227086.643820:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227086.643829:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720679808:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227086.643971:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227086.643983:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720679616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227086.643991:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679744:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227086.705964:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227087.220190:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227087.728150:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227088.240105:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227088.752195:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227089.264045:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227089.780847:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227090.292887:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227090.799991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227091.312148:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227091.763341:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227091.763366:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227091.763370:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227091.763382:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2684 00000100:00080000:0.0:1761227091.763386:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227091.763393:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227091.763406:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227091.763425:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720680064/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227091.763450:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227091.763454:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227091.763488:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227091.763720:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720679872:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227091.763753:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680000:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227091.764113:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720680064:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761227091.769106:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680000:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761227091.769264:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227091.769271:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227091.769276:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720680064:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761227091.769760:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720679872:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227091.769906:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720679936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227091.824653:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227092.339293:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227092.847985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227093.360065:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227093.873059:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227094.384524:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227094.897147:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227095.408034:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227095.922981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227096.432104:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227096.880696:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227096.880704:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227096.880708:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227096.880718:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2690 00000100:00080000:1.0:1761227096.880722:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227096.880728:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227096.880769:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227096.880786:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720680320/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227096.880812:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227096.880816:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227096.880845:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227096.880961:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761227096.880968:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720680128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227096.880993:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227096.881222:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720680320:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227096.886122:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227096.886141:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227096.886849:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720680128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227096.892796:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227096.892809:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227096.892820:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720680320:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227096.944887:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227097.459611:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227097.969325:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227098.480389:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227098.992045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227099.508178:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227100.015993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227100.529996:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227101.040311:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227101.555317:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227102.000154:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227102.000163:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227102.000183:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227102.000194:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2695 00000100:00080000:3.0:1761227102.000197:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227102.000203:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227102.000213:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227102.000229:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000088958979 x1846777720680576/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227102.000252:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227102.000255:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227102.000331:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720680384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227102.000425:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720680512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227102.000457:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227102.000463:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720680448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227102.002943:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720680576:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227102.007220:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720680448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227102.007242:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720680384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227102.007254:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720680512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227102.007466:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227102.007471:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227102.007476:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720680576:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227102.063984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227102.577930:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227103.087999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227103.610121:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227104.112793:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227104.624155:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227105.136002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227105.648436:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227106.159945:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227106.672180:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227107.120149:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227107.120157:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227107.120161:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227107.120189:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2700 00000100:00080000:1.0:1761227107.120193:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227107.120198:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227107.120208:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227107.120224:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720680832/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227107.120245:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227107.120248:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227107.120308:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720680832:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227107.120385:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720680640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227107.120469:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227107.120510:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227107.120522:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227107.128401:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227107.128424:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720680768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227107.128519:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227107.128524:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227107.128539:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720680832:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227107.134269:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720680640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227107.185998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227107.696001:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227108.208278:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227108.725596:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227109.231972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227109.744846:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227110.255982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227110.771832:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227111.280377:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227111.797733:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227112.240985:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227112.240994:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227112.240999:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227112.241013:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2705 00000100:00080000:3.0:1761227112.241016:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227112.241024:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227112.241040:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227112.241062:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abb9f4e4 x1846777720681088/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227112.241096:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227112.241101:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227112.241212:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720680896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227112.241339:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720681024:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227112.241383:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227112.241389:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720680960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227112.242444:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720681088:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227112.245995:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720681024:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227112.247797:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720680960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227112.247815:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720680896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227112.247976:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227112.247981:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227112.247985:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720681088:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227112.304026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227112.818660:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227113.328028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227113.840203:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227114.352009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227114.869496:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227115.375987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227115.891666:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227116.400118:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227116.918897:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227117.360381:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227117.360389:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227117.360395:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227117.360408:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2710 00000100:00080000:1.0:1761227117.360412:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227117.360420:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227117.360433:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227117.360453:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aa6e6fe7 x1846777720681344/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761227117.360699:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720681344:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761227117.360832:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227117.360837:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227117.361229:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720681152:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227117.361271:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720681280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227117.364076:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227117.364102:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720681216:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227117.382054:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227117.382067:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227117.382096:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720681344:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227117.382394:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720681152:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227117.382410:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720681280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227117.390186:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720681216:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227117.424027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227117.936363:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227118.471690:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227118.960112:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227119.472029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227119.984091:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227120.496019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227121.008115:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227121.520573:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227122.032113:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227122.480970:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227122.480980:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227122.480984:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227122.480998:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2715 00000100:00080000:1.0:1761227122.481003:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227122.481011:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227122.481024:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227122.481043:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a2fc13a x1846777720681600/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227122.481070:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227122.481073:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227122.481165:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720681408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227122.481190:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720681600:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227122.481589:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720681536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227122.481703:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227122.481710:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720681472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227122.501716:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227122.501730:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227122.501742:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720681600:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227122.505772:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720681472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227122.505796:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720681408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227122.505809:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720681536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227122.544197:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227123.056700:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227123.569622:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227124.081333:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227124.599120:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227125.104068:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227125.616309:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227126.130011:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227126.640034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227127.155554:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227127.600044:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227127.600052:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227127.600057:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227127.600069:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2720 00000100:00080000:1.0:1761227127.600073:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227127.600080:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227127.600090:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227127.600104:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abc685ce x1846777720681856/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761227127.600205:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720681856:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:1.0:1761227127.600700:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227127.600704:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227127.601342:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720681664:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227127.601409:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720681792:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227127.601441:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227127.601447:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720681728:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227127.605639:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227127.605652:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227127.605663:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720681856:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227127.606500:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720681728:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227127.606514:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720681664:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227127.606523:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720681792:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227127.664628:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227128.190621:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227128.689910:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227129.200170:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227129.713961:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227130.224174:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227130.739821:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227131.248108:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227131.761163:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227132.273061:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227132.723073:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227132.723081:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227132.723085:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227132.723097:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2725 00000100:00080000:3.0:1761227132.723101:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227132.723109:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227132.723119:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227132.723149:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720681920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227132.723155:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d56d40dd x1846777720682112/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227132.723186:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227132.723189:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227132.723253:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720681984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227132.723299:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720682048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227132.723458:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720682112:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761227132.729739:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720681920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227132.730432:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720681984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761227132.738089:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227132.738124:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227132.738139:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720682112:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227132.741781:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720682048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227132.784096:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227133.299322:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227133.807985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227134.321214:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227134.832374:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227135.345128:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227135.857928:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227136.368145:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227136.879998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227137.392593:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227137.840575:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227137.840584:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227137.840588:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227137.840599:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2730 00000100:00080000:1.0:1761227137.840602:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227137.840609:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227137.840621:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227137.840635:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720682368/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227137.840660:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227137.840664:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227137.840775:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720682176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227137.840908:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720682304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227137.840944:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227137.840951:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720682240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227137.841651:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720682368:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227137.852052:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720682240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227137.852072:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720682176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227137.852082:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720682304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227137.852184:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227137.852189:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227137.852195:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720682368:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227137.903988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227138.417602:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227138.928081:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227139.440301:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227139.952633:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227140.464114:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227140.976534:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227141.489199:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227142.000106:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227142.513155:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227142.960368:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227142.960377:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227142.960382:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227142.960395:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2736 00000100:00080000:3.0:1761227142.960399:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227142.960405:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227142.960415:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227142.960430:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720682624/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227142.960455:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227142.960459:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227142.960485:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720682496:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227142.960495:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720682624:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227142.960631:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720682432:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227142.962436:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720682560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227142.963369:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227142.963377:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227142.963382:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720682624:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227142.965567:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720682560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227142.965663:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720682496:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227142.966640:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720682432:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227143.024726:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227143.536972:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227144.048027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227144.560145:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227145.072134:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227145.585736:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227146.096111:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227146.608705:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227147.120611:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227147.638919:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227148.080246:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720682688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761227148.082306:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1761227148.082313:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720682752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761227148.082314:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227148.082318:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227148.082323:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2741 00000100:00080000:0.0:1761227148.082326:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227148.082334:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227148.082347:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227148.082363:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720682880/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227148.082393:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227148.082396:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227148.082419:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227148.082425:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720682816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227148.082527:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720682880:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761227148.091484:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720682752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227148.091561:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720682816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227148.091726:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227148.091732:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227148.091736:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720682880:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227148.091782:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720682688:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227148.143975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227148.658189:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227149.168023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227149.680945:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227150.191991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227150.704049:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227151.215981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227151.728124:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227152.239987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227152.751997:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227153.201588:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227153.201596:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227153.201600:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227153.201610:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2746 00000100:00080000:1.0:1761227153.201613:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227153.201619:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227153.201630:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227153.201643:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094e95ca3 x1846777720683136/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227153.201667:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227153.201670:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227153.201702:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720682944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227153.201712:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720683136:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227153.201766:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720683008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227153.201818:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720683072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227153.212094:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720682944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227153.212317:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227153.212322:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227153.212327:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720683136:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227153.216947:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720683008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227153.217164:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720683072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227153.264527:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227153.776052:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227154.287936:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227154.800560:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227155.311999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227155.823996:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227156.340707:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227156.848039:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227157.359986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227157.872254:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227158.320140:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227158.320149:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227158.320153:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227158.320163:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2751 00000100:00080000:3.0:1761227158.320166:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227158.320172:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227158.320182:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227158.320199:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000727a12f1 x1846777720683392/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227158.320222:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720683200:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227158.320227:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227158.320231:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227158.320271:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720683264:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227158.320314:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720683328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227158.320563:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720683392:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227158.323093:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720683200:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227158.323132:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720683328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227158.324017:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720683264:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227158.324286:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227158.324299:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227158.324304:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720683392:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227158.383967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227158.896369:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227159.408399:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227159.920066:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227160.431992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227160.944162:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227161.455965:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227161.968208:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227162.479987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227162.992170:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227163.440261:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227163.440270:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227163.440274:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227163.440287:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2756 00000100:00080000:3.0:1761227163.440324:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227163.440335:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227163.440350:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227163.440355:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720683456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227163.440375:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000006d85789 x1846777720683648/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227163.440402:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227163.440407:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227163.440453:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720683520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227163.440512:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720683584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227163.440631:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720683648:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227163.445034:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720683520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227163.446549:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720683584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227163.447832:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720683456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761227163.447955:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227163.447960:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227163.447966:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720683648:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227163.504128:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227164.016257:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227164.529189:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227165.043513:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227165.552062:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227166.064559:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227166.576062:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227167.088753:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227167.601021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227168.113345:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227168.560377:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720683712:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227168.568466:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227168.568476:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227168.568481:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227168.568495:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2761 00000100:00080000:3.0:1761227168.568500:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227168.568508:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227168.568522:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227168.568543:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d56d40dd x1846777720683904/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761227168.568648:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720683904:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227168.569565:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720683840:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227168.569629:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227168.569636:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720683776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227168.578007:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227168.578022:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761227168.579311:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227168.579322:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227168.579334:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720683904:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227168.579994:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720683712:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227168.586307:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720683776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227168.586332:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720683840:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227168.630136:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227169.136418:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227169.648074:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227170.160614:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227170.672040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227171.188506:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227171.696025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227172.208108:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227172.720031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227173.232638:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227173.680071:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227173.680080:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227173.680084:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227173.680095:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2766 00000100:00080000:3.0:1761227173.680099:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227173.680105:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227173.680117:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227173.680132:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002b7677b0 x1846777720684160/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227173.680152:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227173.680156:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227173.680240:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720683968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227173.680359:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227173.680392:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227173.680397:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227173.681239:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720684160:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761227173.689033:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227173.689050:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227173.689083:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720684160:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227173.689250:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227173.689264:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720683968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227173.689276:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227173.745627:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227174.256972:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227174.768742:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227175.280108:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227175.791990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227176.304666:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227176.816093:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227177.332495:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227177.840542:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227178.354043:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227178.800388:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227178.800397:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227178.800412:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227178.800423:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2771 00000100:00080000:3.0:1761227178.800452:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227178.800459:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227178.800471:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227178.800489:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720684416/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227178.800515:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227178.800519:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227178.800604:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227178.800713:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761227178.800719:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720684224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227178.800744:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684352:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227178.802717:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720684416:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227178.803779:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720684224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227178.803813:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684352:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227178.803827:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227178.808770:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227178.808785:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227178.808797:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720684416:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227178.868619:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227179.376360:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227179.888501:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227180.400360:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227180.912641:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227181.424305:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227181.935994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227182.448151:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227182.960447:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227183.473115:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227183.920116:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227183.920126:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227183.920130:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227183.920142:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2777 00000100:00080000:3.0:1761227183.920146:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227183.920153:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227183.920164:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227183.920180:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008dec2428 x1846777720684672/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227183.920207:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227183.920210:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227183.920261:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720684672:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761227183.922331:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720684480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227183.922426:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227183.922456:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227183.922462:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:3.0:1761227183.924786:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:3.0:1761227183.924813:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227183.924818:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227183.924842:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720684672:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761227183.930510:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227183.930528:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720684480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227183.930537:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720684608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227183.984051:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227184.496160:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227185.008496:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227185.522377:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227186.032006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227186.549759:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227187.061690:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227187.571563:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227188.080034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227188.594256:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227189.040114:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227189.040122:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227189.040127:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227189.040138:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2782 00000100:00080000:1.0:1761227189.040142:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227189.040148:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227189.040159:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227189.040175:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720684928/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227189.040199:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227189.040203:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227189.040231:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720684800:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227189.040331:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761227189.040337:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720684736:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227189.040362:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720684864:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227189.040706:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720684928:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761227189.044979:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227189.044992:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227189.045003:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720684928:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227189.045097:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720684736:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227189.045111:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720684864:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227189.045120:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720684800:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227189.104040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227189.616295:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227190.127987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227190.640835:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227191.151987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227191.664037:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227192.177098:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227192.688065:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227193.200034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227193.712127:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227194.161377:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227194.161385:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227194.161390:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227194.161401:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2787 00000100:00080000:3.0:1761227194.161404:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227194.161411:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227194.161422:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227194.161438:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c3a840a0 x1846777720685184/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227194.161465:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227194.161468:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227194.161524:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720685184:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227194.162350:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720684992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227194.162829:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720685120:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227194.162910:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227194.162919:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720685056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227194.166455:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227194.166467:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227194.166478:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720685184:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227194.167251:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720685056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227194.167266:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720684992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227194.167276:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720685120:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227194.224318:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227194.736879:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227195.250972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227195.761571:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227196.271979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227196.784054:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227197.296004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227197.808959:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227198.320090:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227198.835758:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227199.280117:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227199.280125:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227199.280129:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227199.280140:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2792 00000100:00080000:2.0:1761227199.280144:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227199.280150:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227199.280161:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227199.280177:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c987228a x1846777720685440/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227199.280216:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227199.280220:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227199.280246:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720685312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227199.280260:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720685440:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227199.280349:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720685248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227199.281435:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720685376:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227199.293199:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227199.293211:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227199.293220:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720685440:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227199.293326:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720685248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227199.293505:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720685312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227199.293578:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720685376:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227199.343985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227199.858009:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227200.367990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227200.880390:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227201.392018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227201.904388:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227202.418311:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227202.928459:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227203.443016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227203.952101:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227204.400182:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227204.400191:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227204.400196:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227204.400210:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2797 00000100:00080000:0.0:1761227204.400215:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227204.400225:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227204.400261:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227204.400284:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7988e14 x1846777720685696/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227204.400308:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227204.400312:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227204.400351:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720685568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227204.400463:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761227204.400470:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720685504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761227204.400497:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720685632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227204.402844:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720685696:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761227204.406643:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720685568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227204.408832:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720685504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227204.410162:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720685632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227204.410294:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227204.410299:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227204.410303:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720685696:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761227204.464514:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227204.976052:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227205.488788:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227206.000154:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227206.511965:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227207.024080:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227207.536641:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227208.048573:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227208.559987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227209.074824:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227209.520348:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227209.520359:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227209.520363:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227209.520373:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2802 00000100:00080000:0.0:1761227209.520376:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227209.520383:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227209.520393:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227209.520409:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720685952/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227209.520435:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227209.520439:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227209.520600:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720685824:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227209.520705:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761227209.520712:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720685760:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227209.520736:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720685888:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227209.524829:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720685952:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761227209.525417:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720685824:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761227209.526061:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720685760:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761227209.529169:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227209.529183:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227209.529193:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720685952:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761227209.531499:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720685888:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227209.583990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227210.096063:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227210.607982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227211.120053:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227211.631979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227212.146149:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227212.656042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227213.168158:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227213.680575:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227214.193840:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227214.640850:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761227214.640960:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720686016:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227214.641153:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720686144:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227214.641196:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227214.641202:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720686080:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227214.641535:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227214.641539:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227214.641546:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2807 00000100:00080000:3.0:1761227214.641550:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227214.641558:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227214.641573:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227214.641597:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009d0a2521 x1846777720686208/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761227214.641653:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720686208:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761227214.642426:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227214.642433:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761227214.647417:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227214.647442:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227214.647452:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720686208:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227214.649782:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720686080:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227214.649801:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720686016:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227214.649811:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720686144:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227214.704238:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227215.216327:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227215.728089:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227216.241099:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227216.752684:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227217.271908:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227217.776029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227218.288185:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227218.801457:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227219.312127:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227219.760657:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227219.760666:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227219.760670:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227219.760683:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2812 00000100:00080000:3.0:1761227219.760688:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227219.760695:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227219.760709:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227219.760730:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c3a840a0 x1846777720686464/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227219.760758:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227219.760761:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227219.760798:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720686464:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761227219.760804:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720686272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227219.761141:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720686400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227219.761182:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227219.761188:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720686336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227219.769502:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720686400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761227219.771134:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227219.771149:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227219.771160:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720686464:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761227219.771285:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720686336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761227219.771298:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720686272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761227219.824360:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227220.337286:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227220.847994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227221.361538:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227221.871959:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227222.385402:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227222.897495:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227223.408103:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227223.923377:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227224.435455:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227224.880080:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227224.880089:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227224.880094:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227224.880106:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2818 00000100:00080000:3.0:1761227224.880109:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227224.880116:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227224.880126:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227224.880142:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8803538 x1846777720686720/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227224.880162:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227224.880166:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227224.880193:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720686592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227224.882174:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720686720:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227224.882324:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720686528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227224.893315:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720686656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227224.897541:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720686592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227224.897650:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227224.897655:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227224.897659:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720686720:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227224.899682:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720686656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227224.901310:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720686528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227224.945669:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227225.459028:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227225.969123:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227226.486383:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227226.993652:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227227.504087:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227228.015988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227228.528097:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227229.040762:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227229.553130:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227230.009130:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720686784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227230.010462:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227230.010468:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227230.010471:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227230.010479:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2823 00000100:00080000:0.0:1761227230.010483:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227230.010489:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227230.010500:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227230.010514:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720686976/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227230.010539:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227230.010542:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227230.010619:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720686848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227230.010691:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227230.010697:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720686912:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227230.010780:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720686976:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227230.017802:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720686784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227230.017988:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227230.017993:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227230.017997:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720686976:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227230.018093:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720686912:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227230.018104:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720686848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227230.063988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227230.579360:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227231.088036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227231.600722:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227232.112449:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227232.624042:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227233.135997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227233.650379:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227234.159987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227234.672459:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227235.121181:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227235.121190:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227235.121195:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227235.121208:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2828 00000100:00080000:3.0:1761227235.121212:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227235.121219:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227235.121230:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227235.121246:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd946368 x1846777720687232/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227235.121266:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227235.121270:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227235.121391:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720687040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227235.121490:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720687168:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227235.121522:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227235.121527:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720687104:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227235.121786:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720687232:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761227235.125662:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227235.125675:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227235.125686:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720687232:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227235.135013:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720687168:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227235.135047:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720687104:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227235.135057:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720687040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227235.184025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227235.696732:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227236.208633:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227236.723082:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227237.231996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227237.744061:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227238.255972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227238.770013:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227239.280109:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227239.792527:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227240.240082:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227240.240090:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227240.240094:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227240.240104:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2833 00000100:00080000:1.0:1761227240.240107:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227240.240113:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227240.240124:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227240.240139:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abc685ce x1846777720687488/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227240.240162:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227240.240165:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227240.240193:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720687360:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227240.240314:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761227240.240321:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720687296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227240.240345:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720687424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227240.240354:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720687488:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761227240.251131:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227240.251142:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227240.251151:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720687488:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227240.251845:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720687360:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227240.251931:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720687296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227240.251942:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720687424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227240.304212:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227240.816972:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227241.330611:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227241.840154:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227242.352240:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227242.864108:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227243.377675:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227243.888055:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227244.399985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227244.917202:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227245.360217:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720687552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227245.361756:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761227245.361764:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720687616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227245.361767:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227245.361775:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227245.361788:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2838 00000100:00080000:3.0:1761227245.361792:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227245.361800:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227245.361813:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227245.361835:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b01aecf x1846777720687744/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227245.361898:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761227245.361901:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:3.0:1761227245.361904:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227245.361914:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720687680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227245.362097:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720687744:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227245.363919:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720687552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227245.366776:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1761227245.366971:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720687616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227245.367001:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720687680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227245.367072:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227245.367083:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720687744:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227245.424517:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227245.936749:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227246.449160:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227246.961124:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227247.474036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227247.984755:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227248.497936:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227249.008427:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227249.520016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227250.032109:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227250.481726:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227250.481736:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227250.481741:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227250.481757:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2843 00000100:00080000:1.0:1761227250.481761:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761227250.481768:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720687808:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227250.481770:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227250.481787:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227250.481831:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a2fc13a x1846777720688000/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227250.481874:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227250.481877:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227250.481921:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720687872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227250.481924:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720687936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227250.482477:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720688000:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227250.488518:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720687808:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227250.489036:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227250.489041:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227250.489046:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720688000:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227250.489919:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720687936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227250.490357:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720687872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227250.544089:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227251.057489:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227251.569219:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227252.082686:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227252.592004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227253.104076:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227253.616743:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227254.134885:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227254.641298:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227255.154985:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227255.600163:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227255.600172:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227255.600176:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227255.600190:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2848 00000100:00080000:3.0:1761227255.600195:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227255.600202:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227255.600218:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227255.600241:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abb9f4e4 x1846777720688256/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227255.600274:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227255.600277:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227255.600978:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720688256:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227255.601193:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720688064:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227255.601232:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227255.601257:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227255.601263:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227255.605907:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227255.605919:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227255.605929:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720688256:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227255.606411:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688128:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227255.606423:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227255.612765:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720688064:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227255.664395:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227256.176449:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227256.688009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227257.200315:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227257.714366:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227258.224322:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227258.736417:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227259.250890:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227259.760342:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227260.273741:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227260.720222:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720688320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227260.721140:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227260.721150:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720688384:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227260.721668:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227260.721675:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227260.721680:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227260.721703:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2853 00000100:00080000:2.0:1761227260.721707:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227260.721714:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227260.721727:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227260.721745:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007221726f x1846777720688512/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761227260.721800:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720688512:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227260.722791:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761227260.722813:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688448:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227260.724801:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227260.724813:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227260.725557:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720688320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227260.729337:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227260.729348:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227260.729360:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720688512:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227260.729988:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720688384:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227260.730307:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688448:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227260.784092:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227261.296282:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227261.808038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227262.323325:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227262.832028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227263.344102:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227263.856494:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227264.368300:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227264.880164:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227265.392071:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227265.840190:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720688576:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227265.842021:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227265.842030:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227265.842035:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227265.842046:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2858 00000100:00080000:2.0:1761227265.842050:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227265.842058:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227265.842073:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227265.842092:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fab9894 x1846777720688768/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227265.842114:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227265.842135:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227265.842203:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720688768:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227265.842692:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720688640:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227265.842751:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227265.842757:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720688704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227265.849580:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227265.849597:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227265.849609:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720688768:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227265.849621:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720688576:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227265.849740:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720688704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227265.849755:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720688640:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227265.904060:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227266.416189:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227266.931416:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227267.440221:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227267.951940:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227268.467781:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227268.975998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227269.488171:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227270.000224:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227270.512243:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227270.960177:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227270.960186:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227270.960191:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227270.960205:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2864 00000100:00080000:3.0:1761227270.960228:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227270.960236:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227270.960250:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227270.960270:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000727a12f1 x1846777720689024/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227270.960297:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227270.960301:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227270.960333:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227270.960451:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761227270.960458:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720688832:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227270.960494:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227270.960530:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720689024:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761227270.965383:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227270.965397:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227270.965409:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720689024:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227270.965589:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720688832:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227270.965650:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227270.975156:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720688960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227271.024168:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227271.536514:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227272.049911:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227272.560314:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227273.072004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227273.585563:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227274.096277:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227274.608248:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227275.121762:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227275.633470:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227276.080197:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227276.080206:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227276.080210:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227276.080223:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2869 00000100:00080000:2.0:1761227276.080226:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227276.080235:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227276.080251:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227276.080271:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000790fc921 x1846777720689280/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227276.080295:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227276.080298:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227276.080333:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720689152:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227276.080391:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720689280:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227276.080473:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761227276.080482:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720689088:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227276.080522:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720689216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227276.085948:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720689088:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227276.085972:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720689216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227276.086004:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720689152:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227276.088435:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227276.088448:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227276.088457:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720689280:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227276.144384:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227276.656200:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227277.167984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227277.680796:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227278.191977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227278.705101:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227279.215990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227279.730308:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227280.239988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227280.752698:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761227281.201047:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720689344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227281.202953:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227281.202960:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227281.202964:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227281.202974:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2874 00000100:00080000:1.0:1761227281.202977:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227281.203000:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227281.203011:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227281.203026:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720689536/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227281.203055:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227281.203058:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227281.203130:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720689408:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227281.203198:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227281.203206:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720689472:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227281.203430:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720689536:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227281.208086:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720689472:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227281.208102:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720689408:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227281.208175:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720689344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227281.208199:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227281.208205:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227281.208210:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720689536:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227281.267662:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227281.777153:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227282.287984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227282.802833:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227283.312122:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227283.824095:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227284.336316:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227284.848888:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227285.359984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227285.872935:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227286.322304:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227286.322314:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227286.322318:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227286.322332:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2879 00000100:00080000:1.0:1761227286.322335:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227286.322342:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227286.322356:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227286.322373:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aa6e6fe7 x1846777720689792/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227286.322412:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227286.322416:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227286.322514:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720689792:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227286.323156:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720689600:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227286.324368:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720689728:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227286.326003:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227286.326024:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720689664:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227286.328010:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720689600:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227286.328024:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227286.328031:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227286.328037:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720689792:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227286.331623:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720689664:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227286.331646:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720689728:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227286.384033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227286.896647:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227287.408031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227287.921189:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227288.432026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227288.948772:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227289.456056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227289.971500:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227290.483760:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227290.998478:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227291.441258:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227291.441269:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227291.441274:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227291.441287:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2884 00000100:00080000:0.0:1761227291.441292:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227291.441300:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227291.441313:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227291.441821:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720689856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227291.441967:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720689984:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227291.442003:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227291.442009:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720689920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227291.450341:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000525c9db8 x1846777720690048/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227291.450382:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227291.450386:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227291.450484:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720690048:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227291.451083:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720689920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227291.451123:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720689984:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227291.459351:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720689856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227291.459411:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227291.459522:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227291.459528:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720690048:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227291.504019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227292.016509:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227292.528051:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227293.043891:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227293.552005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227294.064846:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227294.576437:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227295.088079:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227295.599941:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227296.112140:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227296.560121:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227296.560129:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227296.560133:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227296.560147:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2889 00000100:00080000:0.0:1761227296.560151:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227296.560159:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227296.560172:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227296.560196:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720690304/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227296.560249:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227296.560253:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227296.560396:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720690176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227296.560526:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761227296.560532:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720690112:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227296.560555:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720690240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227296.560895:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720690304:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227296.564545:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720690176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227296.566124:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720690112:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227296.568466:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720690240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227296.570944:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227296.570955:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227296.570965:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720690304:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227296.624786:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227297.136998:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227297.647987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227298.160740:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227298.674903:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227299.184468:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227299.695998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227300.208082:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227300.719976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227301.234687:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227301.680149:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227301.680158:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00100000:0.0:1761227301.680160:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720690368:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227301.680163:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227301.680172:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2894 00000100:00080000:1.0:1761227301.680176:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227301.680184:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227301.680199:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227301.680221:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720690560/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227301.680245:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227301.680249:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227301.680269:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720690432:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227301.680299:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720690496:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227301.680392:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720690560:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227301.683422:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720690368:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227301.686254:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720690432:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227301.687204:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720690496:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227301.687713:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227301.687719:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227301.687724:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720690560:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227301.744245:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227302.256140:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227302.769128:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227303.280089:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227303.792038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227304.304640:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227304.817023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227305.328253:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227305.840148:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227306.353146:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227306.800143:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227306.800150:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227306.800154:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227306.800166:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2899 00000100:00080000:0.0:1761227306.800169:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227306.800177:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227306.800188:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227306.800203:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000243e8b76 x1846777720690816/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227306.800227:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227306.800231:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227306.800267:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720690688:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227306.800347:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720690624:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227306.800400:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720690752:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227306.800601:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720690816:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227306.806180:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720690624:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227306.806263:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720690688:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227306.808896:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720690752:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227306.812065:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227306.812080:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227306.812092:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720690816:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227306.865271:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227307.376204:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227307.888268:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227308.401010:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227308.913516:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227309.426721:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227309.936014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227310.448828:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227310.960102:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227311.474655:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227311.920627:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227311.920636:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227311.920641:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227311.920653:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2905 00000100:00080000:1.0:1761227311.920656:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227311.920663:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227311.920674:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227311.920691:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720691072/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227311.920716:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227311.920719:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227311.920749:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720690944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227311.920902:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761227311.920916:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720690880:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227311.920942:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720691008:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227311.921320:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720691072:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761227311.931263:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227311.931279:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227311.931291:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720691072:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227311.931479:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720690880:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227311.931539:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720691008:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227311.931552:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720690944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227311.984869:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227312.498933:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227313.007938:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227313.521127:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227314.032023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227314.544120:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227315.055961:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227315.568179:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227316.080226:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227316.592061:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227317.040153:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227317.040162:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227317.040166:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227317.040179:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2910 00000100:00080000:3.0:1761227317.040184:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227317.040192:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227317.040205:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227317.040225:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d56d40dd x1846777720691328/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227317.040250:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227317.040254:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227317.040282:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720691200:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227317.040412:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761227317.040419:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720691136:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227317.040478:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720691264:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227317.040615:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720691328:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761227317.047190:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227317.047204:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227317.047215:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720691328:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227317.048795:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720691264:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227317.048813:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720691200:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227317.049281:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720691136:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227317.105648:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227317.616549:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227318.130698:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227318.640105:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227319.152010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227319.664059:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227320.176017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227320.689057:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227321.201019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227321.713581:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227322.161382:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1761227322.161694:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720691392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227322.161804:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720691520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227322.161838:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227322.161844:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720691456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227322.162214:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227322.162219:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227322.162229:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2915 00000100:00080000:0.0:1761227322.162233:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227322.162241:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227322.162256:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227322.162285:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e956e16 x1846777720691584/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227322.162304:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227322.162311:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227322.165543:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720691584:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227322.167975:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720691520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227322.168022:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720691456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227322.170023:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720691392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227322.170361:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227322.170372:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227322.170381:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720691584:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227322.223992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227322.736228:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227323.248066:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227323.760097:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227324.272132:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227324.784388:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227325.296009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227325.808366:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227326.320005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227326.832063:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227327.280836:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227327.280844:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227327.280849:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227327.280887:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2920 00000100:00100000:1.0:1761227327.280904:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720691648:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227327.280907:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227327.280916:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227327.280930:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227327.280948:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720691840/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227327.280958:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227327.280960:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227327.280986:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720691712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227327.281029:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720691776:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227327.281286:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720691840:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227327.283429:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720691648:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227327.283452:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720691776:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227327.283697:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720691712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227327.286888:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227327.286901:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227327.286910:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720691840:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227327.345816:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227327.856512:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227328.368001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227328.880252:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227329.392542:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227329.904246:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227330.416892:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227330.928237:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227331.440004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227331.952096:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227332.400452:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227332.400460:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227332.400465:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227332.400476:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2925 00000100:00080000:3.0:1761227332.400480:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227332.400486:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:2.0:1761227332.400494:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720691904:192.168.202.114@tcp:400:kworker/u8:0.0 00000008:00100000:3.0:1761227332.400498:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227332.400520:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720692096/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227332.400602:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720692032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227332.400612:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720692096:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227332.400641:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227332.400649:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720691968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227332.402326:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227332.402335:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227332.407111:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720691968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227332.407132:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720692032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227332.407234:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227332.407239:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227332.407244:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720692096:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227332.408768:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720691904:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227332.466465:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227332.976927:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227333.488041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227334.001507:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227334.511958:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227335.023994:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227335.536024:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227336.048299:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227336.560511:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227337.072369:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227337.520101:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227337.520108:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227337.520112:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227337.520125:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2930 00000100:00080000:3.0:1761227337.520128:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227337.520135:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227337.520148:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227337.520165:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720692352/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227337.520189:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227337.520193:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227337.520289:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720692352:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227337.521940:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720692160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227337.522016:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720692288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227337.522049:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227337.522055:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720692224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:02020000:0.0:1761227337.526226:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:0.0:1761227337.526265:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227337.526271:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227337.526277:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720692352:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227337.528232:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720692160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227337.528270:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720692288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227337.528300:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720692224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227337.583972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227338.096379:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227338.608353:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227339.120935:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227339.632052:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227340.144836:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227340.656098:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227341.169005:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227341.680031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227342.192026:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227342.640142:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227342.640151:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227342.640155:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227342.640167:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2935 00000100:00080000:1.0:1761227342.640170:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227342.640176:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227342.640196:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227342.640212:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005d7eada1 x1846777720692608/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227342.640241:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227342.640244:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227342.640272:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720692480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227342.640359:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761227342.640365:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720692416:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227342.640388:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720692544:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227342.641773:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720692608:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227342.646791:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720692416:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227342.646818:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720692544:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227342.646849:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720692480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227342.649402:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227342.649415:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227342.649425:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720692608:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227342.704476:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227343.216051:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227343.728661:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227344.240218:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227344.753927:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227345.266135:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227345.775943:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227346.289457:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227346.800219:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227347.312065:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227347.760166:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227347.760174:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227347.760199:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227347.760210:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2940 00000100:00080000:1.0:1761227347.760214:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227347.760221:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227347.760232:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227347.760247:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720692864/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227347.760268:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227347.760271:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227347.760317:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720692672:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227347.760403:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720692800:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227347.760415:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720692864:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227347.760436:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227347.760445:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720692736:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227347.765173:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720692672:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227347.766260:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227347.766266:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227347.766271:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720692864:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227347.770007:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720692736:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227347.770026:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720692800:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227347.823997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227348.338929:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227348.847934:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227349.360153:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227349.872615:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227350.383979:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227350.896048:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227351.408079:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227351.920464:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227352.437325:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227352.880092:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227352.880101:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227352.880106:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227352.880118:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2946 00000100:00080000:0.0:1761227352.880122:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227352.880129:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227352.880141:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227352.880156:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720693120/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227352.880183:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227352.880187:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227352.880252:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720693120:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227352.880636:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720692928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227352.880674:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720693056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227352.880700:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227352.880722:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720692992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227352.890804:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227352.890818:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227352.890828:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720693120:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227352.892611:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720692992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227352.892629:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720692928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227352.892641:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720693056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227352.944378:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227353.456087:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227353.969751:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227354.485282:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227354.992366:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227355.504373:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227356.016008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227356.528241:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227357.040426:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227357.555137:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227358.000082:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227358.000090:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227358.000094:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227358.000105:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2951 00000100:00080000:0.0:1761227358.000107:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227358.000114:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227358.000124:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227358.000139:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000085a527fc x1846777720693376/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227358.000168:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227358.000171:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227358.000226:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720693376:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227358.000316:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720693248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227358.000358:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761227358.000364:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720693184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227358.000385:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720693312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227358.006179:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720693184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227358.006200:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720693248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227358.007712:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227358.007721:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227358.007729:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720693376:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227358.007817:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720693312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227358.068366:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227358.578465:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227359.088011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227359.600177:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227360.112704:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227360.625702:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227361.135990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227361.648020:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227362.160150:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227362.672572:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227363.120130:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227363.120140:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227363.120145:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227363.120158:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2956 00000100:00080000:0.0:1761227363.120163:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227363.120172:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227363.120185:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227363.120208:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720693440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227363.120228:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720693632/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227363.120256:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227363.120260:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227363.120295:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720693504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227363.120331:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720693568:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227363.120471:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720693632:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761227363.125072:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227363.125084:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227363.125094:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720693632:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227363.125488:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720693504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227363.125523:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720693440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227363.125534:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720693568:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227363.183947:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227363.697019:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227364.207991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227364.720166:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227365.231983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227365.744685:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227366.257104:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227366.772806:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227367.280214:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227367.792285:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227368.240433:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1761227368.240441:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720693696:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227368.240442:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227368.240447:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227368.240455:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2961 00000100:00080000:0.0:1761227368.240458:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227368.240465:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227368.240476:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227368.240492:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000805a30b x1846777720693888/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227368.240527:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227368.240531:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227368.240546:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720693824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227368.240580:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227368.240588:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720693760:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227368.240914:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720693888:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227368.246229:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720693696:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227368.246268:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720693760:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227368.246279:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720693824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227368.246399:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227368.246404:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227368.246410:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720693888:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227368.304002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227368.817063:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227369.330041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227369.840055:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227370.351993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227370.864197:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227371.376752:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227371.890278:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227372.400724:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227372.914188:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227373.360069:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227373.360078:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227373.360082:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227373.360092:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2966 00000100:00080000:0.0:1761227373.360112:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227373.360119:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227373.360130:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227373.360147:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a99a8bd5 x1846777720694144/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227373.360170:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227373.360174:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227373.360272:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720694144:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227373.360431:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720693952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227373.360525:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694080:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227373.360568:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227373.360578:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694016:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227373.364625:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227373.364638:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227373.364668:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720694144:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227373.364791:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694016:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227373.364804:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694080:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227373.365323:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720693952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227373.425163:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227373.936360:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227374.449931:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227374.963332:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227375.472319:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227375.984335:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227376.496033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227377.009241:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227377.520034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227378.032063:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227378.484214:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227378.484223:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227378.484227:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227378.484241:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2971 00000100:00080000:0.0:1761227378.484245:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227378.484254:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227378.484269:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227378.484289:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720694400/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227378.484317:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227378.484321:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227378.484357:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694272:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227378.484374:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720694400:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227378.484477:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761227378.484483:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720694208:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227378.484510:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227378.488663:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227378.488679:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227378.488691:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720694400:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227378.488717:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227378.488740:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694272:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227378.492159:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720694208:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227378.544095:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227379.057900:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227379.568032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227380.080485:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227380.594953:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227381.104077:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227381.619223:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227382.128343:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227382.640627:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227383.152127:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227383.600166:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227383.600174:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227383.600178:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227383.600190:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2976 00000100:00080000:1.0:1761227383.600194:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227383.600202:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227383.600215:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227383.600251:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720694656/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227383.600281:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227383.600284:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227383.600317:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720694464:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227383.600442:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720694528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227383.601510:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227383.602610:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720694656:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227383.611188:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720694528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227383.611606:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227383.611613:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227383.611618:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720694656:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227383.611741:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720694464:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227383.611756:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720694592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227383.666817:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227384.176331:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227384.688078:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227385.201065:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227385.712034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227386.225844:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227386.736645:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227387.248064:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227387.760138:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227388.275942:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227388.720203:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227388.720212:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227388.720215:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227388.720227:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2981 00000100:00080000:0.0:1761227388.720231:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227388.720239:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227388.720250:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227388.720270:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720694912/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227388.720288:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227388.720292:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227388.720321:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720694912:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227388.720523:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720694720:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227388.720563:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720694848:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227388.720594:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227388.720601:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720694784:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227388.726041:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227388.726055:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227388.726065:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720694912:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227388.726708:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720694784:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227388.726737:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720694720:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227388.726747:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720694848:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227388.784190:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227389.297678:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227389.808033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227390.320065:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227390.832806:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227391.344077:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227391.856360:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227392.368157:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227392.882927:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227393.392347:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227393.840051:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227393.840061:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227393.840065:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227393.840076:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2986 00000100:00080000:0.0:1761227393.840097:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227393.840103:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227393.840114:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227393.840134:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720695168/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227393.840160:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227393.840163:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227393.840265:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720694976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227393.840265:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720695168:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227393.840352:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720695104:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227393.840439:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227393.840446:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720695040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227393.846200:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227393.846311:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227393.846322:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720695168:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227393.846507:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720694976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227393.846542:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720695104:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227393.846562:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720695040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227393.904925:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227394.416123:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227394.931829:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227395.440165:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227395.952004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227396.464096:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227396.976029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227397.491059:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227398.000900:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227398.512147:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227398.960078:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720695232:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227398.960391:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227398.960400:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720695296:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227398.960696:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227398.960702:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227398.960706:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227398.960711:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2992 00000100:00080000:0.0:1761227398.960715:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227398.960720:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227398.960730:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227398.960746:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e956e16 x1846777720695424/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227398.960772:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227398.960776:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227398.960790:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227398.960798:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720695360:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227398.960829:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720695424:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227398.968489:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720695360:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227398.968548:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720695296:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227398.968697:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720695232:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227398.968807:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227398.968812:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227398.968817:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720695424:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227399.024255:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227399.536251:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227400.048939:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227400.561278:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227401.073839:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227401.585197:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227402.097757:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227402.608063:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227403.120182:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227403.635123:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227404.080198:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227404.080206:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227404.080210:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227404.080228:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 2997 00000100:00080000:0.0:1761227404.080232:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227404.080241:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227404.080264:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227404.080288:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000243e8b76 x1846777720695680/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227404.080318:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1761227404.080318:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720695488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227404.080325:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227404.080365:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720695552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227404.080380:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720695680:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227404.080437:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720695616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227404.083268:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720695488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227404.085326:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720695552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227404.085539:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227404.085545:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227404.085550:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720695680:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227404.086900:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720695616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227404.143990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227404.656355:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227405.168087:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227405.681433:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227406.192036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227406.704166:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227407.216020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227407.728150:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227408.239992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227408.751984:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761227409.200254:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720695744:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227409.200744:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227409.200753:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720695808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227409.201356:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720695872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227409.201792:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227409.201799:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227409.201803:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227409.201809:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3002 00000100:00080000:1.0:1761227409.201812:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227409.201819:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227409.201831:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227409.201851:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720695936/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227409.201898:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227409.201918:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227409.201946:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720695936:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761227409.214021:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227409.214036:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227409.214047:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720695936:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227409.217599:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720695744:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227409.217673:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720695872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227409.217685:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720695808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227409.264030:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227409.776135:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227410.288020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227410.802319:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227411.314526:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227411.824039:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227412.336051:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227412.848164:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227413.363022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227413.874989:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227414.320124:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227414.320133:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227414.320137:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227414.320151:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3007 00000100:00080000:1.0:1761227414.320155:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227414.320163:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227414.320176:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227414.320195:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094e95ca3 x1846777720696192/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227414.320222:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227414.320226:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227414.320337:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720696000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227414.320439:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696128:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227414.320500:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227414.320506:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227414.320805:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720696192:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761227414.322651:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227414.322665:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227414.322693:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720696192:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227414.323617:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227414.323635:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696128:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227414.326918:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720696000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227414.384014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227414.897912:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227415.408043:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227415.920282:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227416.432031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227416.944600:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227417.455977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227417.968713:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227418.480627:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227418.992243:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227419.443263:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227419.443271:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00100000:1.0:1761227419.443272:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720696256:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227419.443275:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227419.443280:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3012 00000100:00080000:0.0:1761227419.443283:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227419.443288:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227419.443297:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227419.443313:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000de75897f x1846777720696448/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761227419.443365:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720696384:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227419.443389:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227419.443393:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720696320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227419.443441:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720696448:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761227419.445686:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227419.445694:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227419.446635:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720696320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227419.446651:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720696384:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227419.446659:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720696256:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227419.447004:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227419.447010:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227419.447016:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720696448:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227419.504031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227420.019286:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227420.528105:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227421.039975:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227421.556665:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227422.064024:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227422.575998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227423.088129:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227423.600009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227424.112129:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227424.560110:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227424.560117:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227424.560122:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227424.560136:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3017 00000100:00080000:0.0:1761227424.560140:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227424.560148:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227424.560162:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227424.560182:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000de75897f x1846777720696704/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227424.560202:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227424.560205:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227424.560427:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720696512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227424.560539:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696640:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227424.560578:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227424.560777:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696576:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227424.560915:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720696704:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761227424.573205:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227424.573225:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227424.573236:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720696704:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227424.573696:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696576:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227424.573732:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720696512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227424.573744:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696640:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227424.623996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227425.139784:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227425.649070:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227426.160137:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227426.672047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227427.185545:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227427.696577:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227428.209809:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227428.720071:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227429.232267:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761227429.680309:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720696768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227429.681334:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227429.681341:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227429.681344:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227429.681352:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3022 00000100:00080000:1.0:1761227429.681356:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227429.681363:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227429.681375:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227429.681390:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720696960/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227429.681447:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720696960:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227429.681537:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227429.681569:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227429.681575:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227429.682357:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227429.682364:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227429.687662:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720696768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227429.688203:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227429.688210:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227429.688215:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720696960:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227429.688322:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227429.688336:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720696832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227429.744112:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227430.256364:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227430.768038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227431.281109:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227431.791983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227432.315714:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227432.815990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227433.328276:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227433.840023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227434.352245:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227434.802736:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227434.802745:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227434.802749:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227434.802774:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3027 00000100:00080000:3.0:1761227434.802778:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227434.802784:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227434.802795:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227434.802812:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000727a12f1 x1846777720697216/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227434.802844:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227434.802848:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227434.802979:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720697216:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227434.803761:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720697024:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227434.803830:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720697152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227434.803886:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227434.803894:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720697088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227434.806586:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227434.806599:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227434.806609:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720697216:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227434.808438:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720697088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227434.808456:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720697024:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227434.808751:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720697152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227434.863985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227435.376889:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227435.888522:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227436.400148:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227436.912027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227437.424290:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227437.938148:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227438.448641:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227438.961044:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227439.472151:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227439.920180:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227439.920188:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227439.920216:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227439.920229:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3033 00000100:00080000:1.0:1761227439.920252:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227439.920260:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227439.920273:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227439.920293:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000094e95ca3 x1846777720697472/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227439.920321:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227439.920325:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227439.920359:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720697344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227439.920457:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720697472:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227439.920468:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761227439.920480:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720697280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227439.920516:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720697408:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227439.925154:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720697280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227439.925183:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720697344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227439.930219:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720697408:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227439.931478:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227439.931489:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227439.931498:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720697472:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227439.984324:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227440.496307:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227441.011611:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227441.520192:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227442.032272:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227442.544333:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227443.056055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227443.569709:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227444.079981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227444.593363:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227445.040400:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227445.040409:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227445.040413:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227445.040424:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3038 00000100:00080000:1.0:1761227445.040428:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227445.040435:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227445.040445:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227445.040481:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000158429d4 x1846777720697728/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227445.040513:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227445.040517:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227445.040610:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720697536:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227445.040704:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720697664:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227445.040737:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227445.040743:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720697600:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227445.041083:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720697728:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227445.047383:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720697600:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227445.047402:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720697536:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227445.047411:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720697664:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227445.047510:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227445.047516:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227445.047520:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720697728:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227445.105943:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227445.616829:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227446.128099:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227446.640724:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227447.152032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227447.666896:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227448.176445:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227448.688725:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227449.200475:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227449.712149:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227450.160517:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227450.160546:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227450.160550:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227450.160561:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3043 00000100:00080000:3.0:1761227450.160565:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227450.160572:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227450.160582:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227450.160603:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abb9f4e4 x1846777720697984/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227450.160604:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720697792:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227450.160737:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720697920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227450.160777:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227450.160783:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720697856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227450.161037:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227450.161043:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227450.161273:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720697984:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761227450.165898:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227450.165909:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227450.165919:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720697984:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227450.165999:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720697856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227450.166022:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720697792:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227450.166059:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720697920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227450.224642:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227450.736342:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227451.251448:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227451.760972:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227452.272016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227452.785369:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227453.297535:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227453.808696:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227454.320016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227454.832970:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227455.281272:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227455.281281:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227455.281285:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227455.281298:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3048 00000100:00080000:1.0:1761227455.281301:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227455.281309:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227455.281320:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227455.281335:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ef613dc x1846777720698240/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227455.281355:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227455.281358:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227455.281389:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720698112:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227455.281501:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761227455.281508:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720698048:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227455.281557:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720698176:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227455.282158:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720698240:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227455.289623:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720698048:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227455.289655:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720698176:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227455.289671:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720698112:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227455.289974:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227455.289979:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227455.289984:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720698240:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227455.344224:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227455.856117:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227456.368537:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227456.880243:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227457.392073:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227457.904036:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227458.416052:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227458.928061:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227459.440059:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227459.953348:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227460.402416:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227460.402426:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227460.402431:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227460.402442:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3053 00000100:00100000:1.0:1761227460.402444:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720698304:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227460.402447:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227460.402458:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227460.402472:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227460.402494:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e956e16 x1846777720698496/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227460.402521:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227460.402544:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227460.402550:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720698432:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227460.402573:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720698368:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227460.403476:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720698496:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227460.412087:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720698368:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227460.414332:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720698304:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227460.414360:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720698432:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227460.415095:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227460.415102:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227460.415108:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720698496:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227460.464332:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227460.976631:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227461.488025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227462.007324:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227462.512046:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227463.024105:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227463.536583:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227464.048303:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227464.560042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227465.072261:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227465.520171:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227465.520179:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227465.520184:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227465.520194:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3058 00000100:00100000:1.0:1761227465.520195:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720698560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227465.520197:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227465.520204:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227465.520218:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227465.520236:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720698752/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227465.520262:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227465.520266:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227465.520282:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720698624:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227465.520298:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720698688:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227465.520314:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720698752:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227465.525174:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720698624:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227465.525255:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720698560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227465.525269:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720698688:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227465.526053:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227465.526060:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227465.526066:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720698752:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227465.584046:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227466.097106:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227466.608618:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227467.120368:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227467.632037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227468.144234:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227468.656158:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227469.168977:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227469.680044:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227470.194175:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227470.640184:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227470.640191:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227470.640194:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227470.640206:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3063 00000100:00080000:3.0:1761227470.640211:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227470.640217:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227470.640229:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227470.640245:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720699008/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227470.640266:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227470.640269:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227470.640347:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720699008:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227470.640634:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720698816:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227470.640668:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720698944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227470.640686:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227470.640690:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720698880:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227470.646027:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227470.646042:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227470.646055:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720699008:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227470.647625:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720698880:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227470.647647:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720698944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227470.648814:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720698816:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227470.706763:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227471.219627:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227471.728623:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227472.244002:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227472.751967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227473.264110:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227473.776746:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227474.291243:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227474.801198:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227475.312187:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227475.761031:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227475.761040:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227475.761044:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227475.761057:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3068 00000100:00100000:1.0:1761227475.761060:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720699072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227475.761061:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227475.761068:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227475.761081:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227475.761098:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720699264/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227475.761122:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227475.761126:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227475.761150:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720699136:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227475.761159:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720699200:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227475.761198:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720699264:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227475.766297:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720699136:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227475.766344:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720699200:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227475.770412:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227475.770425:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227475.770448:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720699264:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227475.771197:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720699072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227475.824355:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227476.338761:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227476.848896:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227477.361226:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227477.872090:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227478.384140:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227478.896168:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227479.409934:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227479.919958:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227480.432073:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227480.883400:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227480.883409:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227480.883413:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227480.883424:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3074 00000100:00080000:0.0:1761227480.883428:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227480.883433:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227480.883443:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227480.883458:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7988e14 x1846777720699520/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227480.883487:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227480.883491:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227480.883593:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720699520:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227480.884908:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720699328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227480.884969:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720699456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227480.885019:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227480.885026:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720699392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227480.890084:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227480.890114:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227480.890125:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720699520:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227480.890377:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720699392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227480.890391:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720699456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227480.892823:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720699328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227480.945127:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227481.456201:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227481.967989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227482.480104:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227482.995303:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227483.504117:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227484.015977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227484.530587:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227485.040394:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227485.552979:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227486.000128:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227486.000136:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227486.000156:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227486.000167:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3079 00000100:00080000:0.0:1761227486.000170:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227486.000177:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227486.000189:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227486.000206:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720699776/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227486.000231:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227486.000234:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227486.000323:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720699776:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227486.002152:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720699584:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227486.002974:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720699712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227486.003019:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227486.003025:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720699648:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227486.004189:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227486.004198:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227486.004206:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720699776:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227486.008355:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720699584:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227486.013700:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720699712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227486.013749:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720699648:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227486.064143:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227486.579007:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227487.087979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227487.600079:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227488.111983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227488.624537:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227489.136628:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227489.648605:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227490.163551:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227490.672060:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227491.120217:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227491.120226:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227491.120230:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227491.120242:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3084 00000100:00080000:1.0:1761227491.120246:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227491.120253:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:0.0:1761227491.120259:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720699840:192.168.202.114@tcp:400:kworker/u8:0.0 00000008:00100000:1.0:1761227491.120264:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227491.120298:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057d87282 x1846777720700032/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227491.120322:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227491.120326:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227491.120344:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720699904:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227491.120370:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720699968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227491.121406:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720700032:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227491.122415:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720699904:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227491.122504:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720699840:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227491.122517:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720699968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:02020000:3.0:1761227491.129941:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:3.0:1761227491.129956:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227491.129962:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227491.129967:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720700032:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227491.184215:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227491.696361:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227492.209021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227492.720131:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227493.234871:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227493.744235:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227494.255984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227494.769804:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227495.279942:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227495.794408:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227496.240805:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227496.240813:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227496.240831:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227496.240842:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3089 00000100:00080000:0.0:1761227496.240845:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227496.240852:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227496.240919:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227496.240937:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000024af3661 x1846777720700288/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227496.240954:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227496.240957:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227496.240980:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227496.241064:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761227496.241071:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720700096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227496.241099:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227496.241163:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720700288:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761227496.247727:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227496.247739:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227496.247750:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720700096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227496.247751:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720700288:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227496.247774:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227496.247786:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700160:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227496.303996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227496.817646:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227497.328413:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227497.842231:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227498.351989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227498.864067:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227499.376753:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227499.888045:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227500.401139:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227500.918120:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227501.360489:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227501.360497:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227501.360500:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227501.360510:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3094 00000100:00080000:3.0:1761227501.360514:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227501.360521:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227501.360531:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227501.360548:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd946368 x1846777720700544/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761227501.360629:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720700544:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227501.360801:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720700352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227501.360840:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720700480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227501.360924:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227501.360932:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720700416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227501.362884:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227501.362894:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761227501.367807:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227501.367837:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227501.367847:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720700544:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227501.369362:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720700416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227501.369380:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720700352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227501.369391:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720700480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227501.424326:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227501.937434:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227502.449039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227502.964610:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227503.472007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227503.987665:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227504.495977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227505.010214:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227505.520150:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227506.032051:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227506.483282:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227506.483290:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227506.483294:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227506.483306:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3099 00000100:00080000:0.0:1761227506.483309:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761227506.483316:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720700608:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227506.483318:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227506.483330:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227506.483346:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720700800/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227506.483370:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227506.483373:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227506.483394:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720700672:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227506.483412:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700736:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227506.483744:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720700800:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227506.491494:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720700608:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227506.491525:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720700672:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227506.491532:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700736:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227506.491603:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227506.491608:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227506.491612:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720700800:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227506.544118:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227507.057263:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227507.569104:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227508.085155:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227508.591971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227509.104268:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227509.616001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227510.131973:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227510.640594:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227511.152201:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227511.600151:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227511.600159:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227511.600163:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227511.600175:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3104 00000100:00080000:0.0:1761227511.600178:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227511.600185:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227511.600199:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227511.600220:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720701056/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227511.600242:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227511.600246:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227511.600271:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700928:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227511.600363:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761227511.600369:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720700864:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227511.600408:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700992:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227511.604700:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720701056:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227511.605330:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720700864:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227511.607245:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700992:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227511.607262:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720700928:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227511.610690:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227511.610724:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227511.610734:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720701056:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227511.663993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227512.176099:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227512.688047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227513.200162:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227513.712741:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227514.225212:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227514.737073:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227515.248315:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227515.761305:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227516.275062:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227516.720049:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227516.720057:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227516.720061:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227516.720073:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3109 00000100:00080000:0.0:1761227516.720076:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227516.720082:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227516.720092:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227516.720107:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d88fcfe6 x1846777720701312/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761227516.720119:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701184:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227516.720184:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720701312:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227516.720213:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761227516.720220:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720701120:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227516.720239:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701248:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227516.720293:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227516.720297:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227516.725806:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720701120:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227516.725831:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701248:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227516.725840:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701184:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227516.725926:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227516.725936:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227516.725946:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720701312:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227516.783995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227517.296190:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227517.810056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227518.320680:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227518.832017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227519.344751:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227519.856569:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227520.369251:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227520.880434:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227521.392110:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227521.840307:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227521.840318:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227521.840323:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761227521.840326:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720701440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227521.840336:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3114 00000100:00080000:0.0:1761227521.840341:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227521.840349:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227521.840370:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227521.840405:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000805a30b x1846777720701568/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227521.840430:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227521.840435:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227521.840463:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720701376:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227521.840500:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720701568:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227521.840551:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227521.848697:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227521.848711:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227521.848725:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720701568:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227521.849718:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227521.849769:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720701440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227521.850281:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720701376:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227521.907682:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227522.416114:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227522.928016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227523.441292:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227523.954938:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227524.465890:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227524.976306:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227525.491733:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227526.000024:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227526.512144:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227526.960107:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227526.960115:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227526.960119:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227526.960148:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3120 00000100:00080000:0.0:1761227526.960151:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227526.960159:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227526.960171:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227526.960191:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000024af3661 x1846777720701824/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227526.960216:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227526.960219:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227526.960253:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227526.960393:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761227526.960399:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720701632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227526.960422:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227526.960684:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720701824:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227526.965681:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720701632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227526.970705:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227526.971901:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227526.971911:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227526.971918:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720701824:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227526.973297:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720701760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227527.026468:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227527.537090:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227528.047997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227528.561093:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227529.072094:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227529.587001:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227530.097432:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227530.608216:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227531.120007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227531.634014:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227532.081414:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227532.081421:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227532.081425:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227532.081434:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3125 00000100:00080000:0.0:1761227532.081438:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227532.081443:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227532.081452:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227532.081466:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720702080/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227532.081522:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720702080:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761227532.082916:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227532.082923:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227532.082953:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720701952:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227532.083009:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761227532.083015:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720701888:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227532.083037:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702016:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227532.085719:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720701888:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227532.085760:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702016:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227532.085770:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720701952:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227532.089000:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227532.089011:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227532.089021:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720702080:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227532.143973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227532.656794:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227533.167974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227533.683980:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227534.192035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227534.704038:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227535.215994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227535.729519:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227536.240568:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227536.755700:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227537.200389:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227537.200397:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227537.200405:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227537.200425:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3130 00000100:00080000:0.0:1761227537.200429:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227537.200436:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227537.200450:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227537.200466:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7988e14 x1846777720702336/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227537.200492:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227537.200495:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227537.200541:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720702208:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227537.200542:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720702336:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227537.202922:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720702144:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227537.203476:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702272:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227537.209198:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720702208:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227537.209401:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227537.209406:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227537.209411:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720702336:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227537.209560:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702272:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227537.214618:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720702144:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227537.264039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227537.778253:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227538.287977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227538.801716:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227539.311989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227539.825181:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227540.336449:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227540.848180:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227541.360049:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227541.872350:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227542.320097:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227542.320105:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227542.320110:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227542.320122:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3135 00000100:00080000:0.0:1761227542.320127:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1761227542.320135:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227542.320174:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227542.320196:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720702592/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227542.320220:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227542.320224:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227542.320258:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702464:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227542.320368:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761227542.320375:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720702400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227542.320398:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702528:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227542.320720:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720702592:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227542.324670:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720702400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227542.335236:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702464:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227542.335276:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702528:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227542.335646:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227542.335651:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227542.335656:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720702592:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227542.384022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227542.897632:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227543.408023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227543.922539:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227544.432009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227544.944166:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227545.456036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227545.969806:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227546.480033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227546.992150:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227547.440178:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720702656:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227547.441550:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720702720:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227547.441820:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227547.441826:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720702784:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227547.442073:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227547.442079:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227547.442083:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227547.442090:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3140 00000100:00080000:0.0:1761227547.442103:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1761227547.442109:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227547.442120:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227547.442137:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fe9fa97 x1846777720702848/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227547.442159:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227547.442163:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227547.442219:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720702848:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227547.449578:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720702656:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227547.451537:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720702784:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227547.451559:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720702720:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227547.458530:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227547.458544:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227547.458555:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720702848:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227547.504002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227548.016893:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227548.530559:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227549.040102:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227549.552562:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227550.065334:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227550.577117:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227551.091772:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227551.599983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227552.112571:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227552.562342:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227552.562352:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227552.562356:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227552.562387:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3145 00000100:00080000:1.0:1761227552.562391:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1761227552.562398:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227552.562410:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227552.562421:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720702912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227552.562431:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720703104/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227552.562456:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227552.562460:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227552.562516:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720703104:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227552.562520:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227552.562550:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720703040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227552.567563:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720702976:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227552.567691:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720703040:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227552.570835:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227552.570851:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227552.570902:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720703104:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227552.572767:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720702912:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227552.623996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227553.137916:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227553.652566:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227554.160355:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227554.672056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227555.185759:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227555.696008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227556.208249:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227556.720836:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227557.236103:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227557.680814:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227557.680823:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227557.680828:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227557.680841:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3150 00000100:00080000:1.0:1761227557.680847:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00100000:0.0:1761227557.680851:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720703168:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227557.680988:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227557.681034:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:0.0:1761227557.681038:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227557.681046:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703232:192.168.202.114@tcp:400:kworker/u8:0.0 00000008:00100000:1.0:1761227557.681051:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227557.681077:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720703360/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227557.681134:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720703360:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:1.0:1761227557.681947:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227557.681953:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227557.684290:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703232:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227557.684312:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720703168:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227557.684324:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227557.684659:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227557.684666:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227557.684671:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720703360:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227557.744552:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227558.256191:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227558.768703:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227559.280877:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227559.792037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227560.304481:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227560.816529:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227561.328241:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227561.842894:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227562.352043:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227562.800075:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227562.800105:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227562.800113:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227562.800130:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3155 00000100:00080000:1.0:1761227562.800136:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227562.800144:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:0.0:1761227562.800156:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720703424:192.168.202.114@tcp:400:kworker/u8:0.0 00000008:00100000:1.0:1761227562.800160:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227562.800188:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720703616/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227562.800215:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227562.800219:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227562.800279:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227562.800292:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720703552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227562.800293:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720703616:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227562.806229:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720703424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227562.806259:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720703552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227562.806584:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227562.806594:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227562.806604:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720703616:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227562.807005:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227562.864269:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227563.376507:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227563.889623:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227564.402697:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227564.912038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227565.425831:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227565.936005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227566.448014:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227566.960188:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227567.474311:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227567.921058:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227567.921069:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227567.921074:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227567.921089:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3161 00000100:00100000:1.0:1761227567.921091:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720703680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227567.921094:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227567.921106:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227567.921121:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227567.921144:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720703872/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227567.921170:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227567.921174:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227567.921210:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227567.921252:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227567.921265:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703744:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227567.923756:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720703872:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227567.926791:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720703680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227567.937039:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227567.942747:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720703744:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227567.943998:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227567.944006:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227567.944012:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720703872:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227567.984007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227568.495992:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227569.008011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227569.522353:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227570.033630:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227570.549040:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227571.056156:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227571.568963:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227572.080451:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227572.592275:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227573.040036:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227573.040046:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227573.040051:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227573.040063:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3166 00000100:00080000:0.0:1761227573.040066:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227573.040072:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227573.040083:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227573.040120:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720704128/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227573.040197:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720704128:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227573.040444:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227573.040490:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761227573.040505:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720703936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227573.040530:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227573.041086:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227573.041090:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227573.048370:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720703936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227573.048402:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227573.048457:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227573.048463:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227573.048469:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720704128:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227573.048988:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227573.104032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227573.618279:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227574.129639:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227574.641284:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227575.151994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227575.667208:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227576.176047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227576.691119:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227577.200521:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227577.712601:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227578.162018:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227578.162028:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227578.162033:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227578.162047:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3171 00000100:00080000:0.0:1761227578.162051:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227578.162062:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:1.0:1761227578.162066:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720704192:192.168.202.114@tcp:400:kworker/u8:0.0 00000008:00100000:0.0:1761227578.162077:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227578.162099:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fe9fa97 x1846777720704384/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227578.162124:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227578.162128:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227578.162151:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720704256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227578.162181:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227578.163365:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720704384:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227578.170925:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720704256:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227578.170978:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227578.171613:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227578.171619:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227578.171624:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720704384:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227578.171709:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720704192:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227578.224315:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227578.739124:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227579.250009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227579.760147:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227580.271992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227580.784423:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227581.296256:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227581.808062:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227582.320026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227582.832298:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227583.280081:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227583.280090:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227583.280095:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227583.280130:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3176 00000100:00080000:0.0:1761227583.280134:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227583.280141:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227583.280156:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227583.280175:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018849446 x1846777720704640/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227583.280200:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227583.280203:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227583.280298:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720704640:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227583.282386:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720704448:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227583.282509:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720704576:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227583.282544:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227583.282551:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720704512:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227583.288775:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227583.288786:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227583.288795:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720704640:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227583.289522:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720704512:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227583.289538:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720704576:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227583.290044:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720704448:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227583.344594:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227583.856173:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227584.368027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227584.880504:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227585.392125:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227585.904089:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227586.416511:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227586.929003:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227587.440881:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227587.954194:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227588.400054:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227588.400064:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227588.400068:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227588.400080:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3181 00000100:00080000:0.0:1761227588.400082:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227588.400089:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227588.400099:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227588.400115:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720704896/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227588.400139:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227588.400142:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227588.400240:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720704896:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227588.403719:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720704704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227588.405023:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227588.405103:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227588.405111:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227588.405694:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227588.405705:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227588.405715:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720704896:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227588.412048:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227588.412100:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720704832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227588.415519:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720704704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227588.464968:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227588.977798:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227589.489391:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227590.000022:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227590.512503:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227591.024060:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227591.535936:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227592.048164:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227592.560277:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227593.072240:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227593.520075:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720704960:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227593.520271:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227593.520276:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720705024:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227593.521883:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227593.521891:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227593.521894:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227593.521904:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3186 00000100:00080000:0.0:1761227593.521907:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227593.521914:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227593.521927:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227593.521944:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000085a527fc x1846777720705152/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227593.521973:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227593.521976:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227593.522030:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720705088:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227593.522200:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720705152:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761227593.534027:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227593.534040:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227593.534051:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720705152:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227593.534204:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720705024:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227593.534227:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720705088:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227593.543326:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720704960:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227593.583978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227594.100050:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227594.607986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227595.120047:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227595.634711:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227596.145485:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227596.657002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227597.168878:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227597.680087:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227598.193145:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227598.640443:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227598.640452:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227598.640456:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1761227598.640458:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720705216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227598.640481:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3191 00000100:00080000:0.0:1761227598.640485:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227598.640491:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227598.640503:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227598.640523:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000024af3661 x1846777720705408/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227598.640551:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1761227598.640551:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720705344:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227598.640557:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227598.640590:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720705280:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227598.640801:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720705408:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227598.644701:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720705216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227598.645079:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720705280:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227598.646518:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227598.646526:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227598.646534:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720705408:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227598.646810:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720705344:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227598.703976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227599.216044:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227599.728058:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227600.240120:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227600.752108:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227601.266145:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227601.777005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227602.291592:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227602.800686:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227603.314507:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227603.760171:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227603.760180:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227603.760185:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227603.760200:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3196 00000100:00080000:0.0:1761227603.760205:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227603.760212:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227603.760226:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227603.760248:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720705664/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761227603.760324:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720705664:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227603.760332:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720705472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227603.760369:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227603.760373:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227603.760494:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720705536:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227603.760555:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720705600:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227603.766631:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720705472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227603.774641:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227603.774654:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227603.774665:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720705664:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227603.774778:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720705536:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227603.774809:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720705600:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227603.823989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227604.336126:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227604.847990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227605.362084:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227605.871971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227606.385105:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227606.895982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227607.408051:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227607.919991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227608.432080:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227608.880049:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227608.880079:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720705728:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227608.880118:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761227608.880131:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720705792:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227608.880906:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227608.880912:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227608.880916:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227608.880922:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3202 00000100:00080000:2.0:1761227608.880925:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761227608.880930:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227608.880940:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227608.880952:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720705856:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227608.880955:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000003513afe x1846777720705920/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227608.880979:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227608.880982:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227608.881014:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720705920:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761227608.883628:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227608.883642:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227608.883651:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720705920:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227608.883739:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720705728:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227608.883794:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720705856:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227608.883804:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720705792:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227608.944014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227609.458092:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227609.968300:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227610.480636:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227610.992011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227611.504150:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227612.015990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227612.532997:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227613.039996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227613.553454:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761227614.001651:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720705984:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227614.001804:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720706048:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227614.001893:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227614.001899:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227614.001903:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227614.001926:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3207 00000100:00080000:0.0:1761227614.001930:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227614.001936:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:0.0:1761227614.001949:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720706112:192.168.202.114@tcp:400:kworker/u8:3.0 00000008:00100000:0.0:1761227614.001998:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227614.002017:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000805a30b x1846777720706176/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227614.002035:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227614.002038:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227614.002225:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720706176:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227614.022046:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720706112:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227614.022157:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227614.022162:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227614.022167:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720706176:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227614.022305:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720706048:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227614.023715:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720705984:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227614.064265:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227614.576606:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227615.087984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227615.601083:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227616.112007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227616.625203:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227617.135974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227617.648201:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227618.160897:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227618.679498:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227619.120140:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227619.120150:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227619.120154:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227619.120166:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3212 00000100:00080000:1.0:1761227619.120169:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227619.120175:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227619.120200:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227619.120217:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720706432/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227619.120237:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227619.120240:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227619.120267:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720706304:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227619.120355:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761227619.120361:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720706240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227619.120418:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720706368:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227619.120918:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720706432:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227619.126880:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720706240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227619.129382:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720706304:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227619.129418:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720706368:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227619.134978:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227619.134989:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227619.134999:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720706432:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227619.189594:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227619.696094:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227620.209307:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227620.721479:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227621.233611:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227621.744174:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227622.256935:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227622.770517:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227623.280576:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227623.792684:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227624.240079:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227624.240088:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227624.240092:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227624.240104:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3217 00000100:00080000:0.0:1761227624.240108:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227624.240131:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227624.240143:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227624.240161:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000805a30b x1846777720706688/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227624.240191:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227624.240194:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227624.240221:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720706560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227624.240329:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761227624.240337:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720706496:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227624.240361:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720706624:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227624.244021:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720706496:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227624.244047:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720706624:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227624.244058:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720706560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227624.244233:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720706688:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761227624.247825:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227624.247843:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227624.247888:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720706688:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227624.304438:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227624.816944:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227625.329475:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227625.843355:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227626.352969:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227626.865230:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227627.377368:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227627.889717:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227628.400004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227628.914818:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227629.360670:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227629.360678:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227629.360683:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227629.360694:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3222 00000100:00080000:0.0:1761227629.360697:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227629.360704:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227629.360714:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227629.360730:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720706944/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227629.360752:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227629.360755:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227629.360799:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720706816:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227629.360946:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720706752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227629.360999:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720706944:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227629.362021:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720706880:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227629.370273:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720706816:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227629.370326:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720706752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227629.370337:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720706880:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227629.371818:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227629.371827:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227629.371835:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720706944:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227629.424239:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227629.937328:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227630.448480:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227630.960149:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227631.471958:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227631.984005:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227632.497353:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227633.008815:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227633.520082:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227634.036065:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227634.480098:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227634.480107:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227634.480111:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227634.480123:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3227 00000100:00080000:0.0:1761227634.480126:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227634.480133:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227634.480145:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227634.480161:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d991e6ad x1846777720707200/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227634.480189:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227634.480193:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227634.480342:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720707008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227634.480438:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720707136:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227634.480474:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227634.480480:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720707072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227634.480579:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720707200:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761227634.491575:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1761227634.491586:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720707008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227634.491589:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227634.491597:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720707200:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227634.491609:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720707136:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227634.491655:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720707072:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227634.544900:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227635.056005:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227635.570897:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227636.080117:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227636.598474:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227637.104232:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227637.616279:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227638.128060:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227638.640626:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227639.155013:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227639.601730:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227639.601739:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227639.601743:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227639.601767:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3232 00000100:00080000:2.0:1761227639.601771:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761227639.601795:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227639.601806:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227639.601823:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084e78951 x1846777720707456/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227639.601850:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227639.601883:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227639.601920:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720707328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227639.602030:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761227639.602037:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720707264:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227639.602066:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720707392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227639.602271:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720707456:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227639.606783:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720707264:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227639.606809:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720707392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227639.606820:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720707328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227639.606950:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227639.606955:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227639.606960:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720707456:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227639.666286:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227640.178901:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227640.688022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227641.200927:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227641.712374:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227642.225121:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227642.737702:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227643.248731:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227643.760125:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227644.275559:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227644.722335:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720707520:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227644.722578:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227644.722585:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720707584:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227644.723007:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227644.723014:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227644.723017:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227644.723023:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3237 00000100:00080000:0.0:1761227644.723026:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227644.723033:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:1.0:1761227644.723222:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227644.723231:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720707648:192.168.202.114@tcp:400:kworker/u8:3.0 00000008:00100000:0.0:1761227644.723344:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227644.723367:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720707712/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227644.723417:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227644.723421:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227644.723810:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720707712:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227644.725822:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720707520:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227644.726085:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720707584:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227644.733240:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720707648:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:02020000:2.0:1761227644.733789:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:2.0:1761227644.755848:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227644.755896:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227644.755911:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720707712:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227644.784028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227645.296275:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227645.807996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227646.320156:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227646.832168:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227647.344013:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227647.855996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227648.370701:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227648.881829:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227649.392164:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227649.840190:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227649.840227:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227649.840232:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227649.840245:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3242 00000100:00080000:2.0:1761227649.840249:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761227649.840256:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227649.840270:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227649.840289:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720707968/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227649.840317:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227649.840320:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227649.840355:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720707840:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227649.840361:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720707968:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227649.840552:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720707776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227649.840740:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720707904:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227649.843344:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227649.843359:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227649.843370:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720707968:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227649.845403:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720707776:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227649.846217:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720707840:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227649.846399:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720707904:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227649.904113:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227650.416933:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227650.928107:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227651.446333:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227651.951919:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227652.465646:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227652.976101:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227653.489152:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227654.001783:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227654.512081:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227654.960172:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227654.960181:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227654.960185:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227654.960214:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3248 00000100:00080000:1.0:1761227654.960218:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227654.960224:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227654.960234:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227654.960251:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720708224/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227654.960279:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227654.960282:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227654.960367:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720708224:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227654.960820:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720708032:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227654.960946:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720708160:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227654.960975:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227654.960980:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720708096:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227654.966803:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227654.966816:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227654.966826:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720708224:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227654.966991:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720708032:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227654.967850:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720708160:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227654.969292:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720708096:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227655.025426:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227655.536780:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227656.047985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227656.562925:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227657.073138:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227657.584071:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227658.096055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227658.612889:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227659.120053:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227659.633124:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227660.080136:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227660.080144:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227660.080148:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227660.080162:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3253 00000100:00080000:1.0:1761227660.080166:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227660.080174:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227660.080188:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227660.080207:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dc5f36b1 x1846777720708480/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227660.080237:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227660.080241:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227660.080353:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720708288:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227660.080462:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720708416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227660.080498:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227660.080504:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720708352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227660.080928:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720708480:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761227660.085435:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227660.085449:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227660.085460:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720708480:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227660.085579:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720708352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227660.085592:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720708416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227660.086305:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720708288:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227660.144456:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227660.661697:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227661.168387:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227661.682591:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227662.192278:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227662.705474:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227663.216041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227663.731052:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227664.240004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227664.752117:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227665.200171:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227665.200179:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227665.200183:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227665.200194:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3258 00000100:00080000:0.0:1761227665.200197:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227665.200205:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227665.200215:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227665.200234:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018849446 x1846777720708736/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227665.200260:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227665.200264:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227665.200301:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720708608:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227665.200393:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761227665.200400:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720708544:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227665.200424:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720708672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227665.200769:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720708736:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227665.209039:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720708544:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227665.209082:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720708672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227665.209094:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720708608:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227665.212482:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227665.212493:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227665.212502:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720708736:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227665.263976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227665.776170:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227666.288110:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227666.801309:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227667.312018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227667.824146:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227668.336066:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227668.849467:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227669.362170:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227669.872328:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227670.321577:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227670.321586:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227670.321589:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227670.321601:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3263 00000100:00080000:2.0:1761227670.321606:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761227670.321612:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227670.321640:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227670.321657:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072dbf25b x1846777720708992/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761227670.321739:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720708992:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761227670.325310:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227670.325321:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227670.325700:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720708800:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227670.325808:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720708928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227670.325843:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227670.325881:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720708864:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227670.329205:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227670.329218:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227670.329247:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720708992:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227670.333323:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720708864:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227670.333343:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720708800:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227670.333353:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720708928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227670.383994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227670.896097:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227671.408141:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227671.920206:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227672.432027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227672.944796:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227673.456985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227673.968374:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227674.480032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227674.994624:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227675.441068:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227675.441076:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227675.441080:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227675.441093:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3268 00000100:00080000:0.0:1761227675.441096:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227675.441104:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227675.441116:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227675.441134:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720709248/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227675.441161:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227675.441165:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227675.441201:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709120:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227675.441311:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761227675.441317:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720709056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227675.441343:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227675.441595:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720709248:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227675.444549:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720709056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227675.445579:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227675.445588:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227675.445595:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720709248:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227675.448008:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709120:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227675.448047:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227675.504435:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227676.017426:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227676.528833:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227677.046908:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227677.552037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227678.064124:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227678.583906:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227679.088104:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227679.600244:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227680.114830:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227680.560347:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720709312:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227680.563605:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227680.563625:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709376:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227680.564331:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227680.564338:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227680.564342:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227680.564348:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3273 00000100:00080000:0.0:1761227680.564366:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227680.564373:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227680.564384:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227680.564401:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720709504/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227680.564424:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227680.564428:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227680.564522:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227680.564536:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720709440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227680.565361:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720709504:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227680.566130:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720709312:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227680.570082:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720709440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227680.570136:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709376:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227680.571083:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227680.571094:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227680.571102:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720709504:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227680.623979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227681.136043:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227681.648208:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227682.161972:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227682.672329:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227683.187552:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227683.696002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227684.208297:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227684.720192:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227685.236455:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227685.680222:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227685.680232:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227685.680237:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227685.680251:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3278 00000100:00080000:0.0:1761227685.680255:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761227685.680263:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227685.680277:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227685.680297:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720709568:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227685.680303:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720709760/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227685.680388:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720709760:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227685.680416:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720709696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227685.680467:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227685.680475:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720709632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227685.680486:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227685.680492:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227685.683496:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720709632:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227685.683517:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720709568:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227685.683549:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720709696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227685.688087:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227685.688111:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227685.688119:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720709760:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227685.744782:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227686.257260:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227686.767916:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227687.280176:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227687.791978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227688.305963:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227688.816032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227689.328227:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227689.843660:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227690.354979:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227690.800621:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227690.800631:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227690.800635:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227690.800648:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3283 00000100:00100000:0.0:1761227690.801672:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720709824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227690.801791:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227690.801829:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227690.801836:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709888:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227690.802334:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761227690.811424:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227690.811446:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227690.811489:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720710016/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227690.811518:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227690.811522:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227690.814485:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720710016:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227690.818198:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227690.822462:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720709888:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227690.822484:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720709824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227690.824637:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227690.824649:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227690.824659:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720710016:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227690.865131:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227691.379255:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227691.888870:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227692.404660:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227692.915192:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227693.424921:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227693.935997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227694.448076:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227694.960921:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227695.473723:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227695.920050:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227695.920060:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227695.920064:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227695.920076:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3289 00000100:00100000:0.0:1761227695.920077:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720710144:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227695.920080:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227695.920089:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227695.920099:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227695.920116:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000158429d4 x1846777720710272/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227695.920143:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227695.920147:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227695.920184:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761227695.920206:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720710080:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227695.920233:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720710208:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227695.920350:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720710272:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227695.927732:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720710144:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227695.931884:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720710080:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227695.932320:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227695.932326:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227695.932331:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720710272:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227695.933503:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720710208:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227695.983990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227696.497849:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227697.008004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227697.521385:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227698.032151:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227698.544766:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227699.056411:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227699.568654:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227700.080052:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227700.592179:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227701.042071:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227701.042080:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227701.042090:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227701.042103:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3294 00000100:00080000:0.0:1761227701.042107:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227701.042116:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227701.042147:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227701.042167:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000243e8b76 x1846777720710528/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227701.042208:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227701.042212:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227701.042255:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720710400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227701.042365:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761227701.042371:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720710336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227701.042397:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720710464:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227701.047006:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720710528:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227701.048094:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720710336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227701.053499:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227701.053512:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227701.053523:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720710528:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227701.053747:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720710400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227701.053768:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720710464:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227701.104126:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227701.616178:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227702.127992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227702.640827:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227703.152098:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227703.664115:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227704.179552:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227704.688140:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227705.199989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227705.712588:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761227706.160113:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720710592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227706.160113:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227706.160123:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227706.160129:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227706.160135:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3299 00000100:00080000:1.0:1761227706.160139:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227706.160146:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227706.160157:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227706.160173:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720710784/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227706.160206:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227706.160209:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227706.160230:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720710720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227706.160240:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720710656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227706.160252:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720710784:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227706.165996:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720710592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227706.166660:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720710720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227706.175154:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227706.175167:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227706.175178:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720710784:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227706.175378:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720710656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227706.223977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227706.737692:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227707.248010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227707.765288:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227708.271973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227708.785039:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227709.296445:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227709.808114:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227710.320598:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227710.832707:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227711.280087:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227711.280096:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227711.280100:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227711.280113:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3304 00000100:00080000:1.0:1761227711.280117:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227711.280127:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227711.280141:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227711.280178:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aa6e6fe7 x1846777720711040/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227711.280209:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227711.280213:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227711.280247:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720710912:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227711.280267:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720710848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227711.280381:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720711040:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227711.280385:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720710976:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227711.283186:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720710976:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227711.283240:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720710912:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227711.285137:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227711.285148:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227711.285156:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720711040:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227711.285269:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720710848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227711.344064:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227711.859793:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227712.368574:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227712.880797:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227713.392066:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227713.904479:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227714.416038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227714.929373:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227715.440090:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227715.954552:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227716.400700:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227716.400708:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227716.400713:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227716.400722:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3309 00000100:00100000:3.0:1761227716.400723:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720711104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227716.400726:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227716.400735:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227716.400748:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227716.400764:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720711296/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227716.400795:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227716.400799:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227716.400830:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720711232:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227716.400843:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720711168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227716.402268:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720711296:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227716.412332:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720711104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227716.412821:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720711168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227716.412871:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227716.412876:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227716.412965:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720711296:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227716.413057:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720711232:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227716.463981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227716.979381:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227717.490165:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227717.999996:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227718.515816:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227719.024227:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227719.536737:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227720.050161:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227720.560599:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227721.073644:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227721.520084:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227721.520093:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227721.520097:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227721.520109:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3314 00000100:00080000:1.0:1761227721.520113:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227721.520120:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227721.520130:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227721.520147:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720711552/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227721.520169:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227721.520173:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227721.520319:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720711360:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227721.520415:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720711488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227721.520460:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227721.520465:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720711424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227721.521696:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720711552:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227721.527946:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720711360:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227721.531276:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720711424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227721.531299:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720711488:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227721.534760:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227721.534775:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227721.534787:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720711552:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227721.584198:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227722.096040:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227722.608040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227723.125314:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227723.631993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227724.144358:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227724.655973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227725.168149:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227725.680141:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227726.192254:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227726.644711:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227726.644724:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227726.644728:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227726.644745:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3319 00000100:00080000:2.0:1761227726.644749:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227726.644756:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227726.644769:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227726.644785:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b3214ada x1846777720711808/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227726.644811:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227726.644814:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227726.644915:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720711808:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227726.645792:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720711616:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227726.645903:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720711744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227726.645949:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227726.645956:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720711680:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227726.649603:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227726.649617:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227726.649630:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720711808:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227726.649775:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720711680:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227726.649789:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720711616:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227726.649801:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720711744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227726.704035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227727.216250:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227727.729357:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227728.240451:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227728.752116:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227729.265323:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227729.775974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227730.289170:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227730.800672:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227731.313504:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227731.760243:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720711872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227731.761784:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761227731.761801:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720711936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227731.762280:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227731.762286:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227731.762290:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227731.762296:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3324 00000100:00080000:2.0:1761227731.762300:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227731.762306:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227731.762336:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227731.762356:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720712064/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227731.762369:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227731.762372:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227731.762573:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720712064:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227731.762664:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720712000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227731.774336:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720711872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227731.776221:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720712000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227731.777517:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227731.777530:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227731.777541:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720712064:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227731.777982:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720711936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227731.824246:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227732.339900:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227732.848043:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227733.360157:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227733.872147:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227734.386724:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227734.896059:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227735.408248:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227735.921578:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227736.432198:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227736.880306:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227736.880315:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227736.880320:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227736.880332:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3330 00000100:00080000:0.0:1761227736.880336:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227736.880345:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227736.880360:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227736.880380:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000525c9db8 x1846777720712320/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227736.880466:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720712320:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227736.884034:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720712192:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227736.884155:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761227736.884164:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720712128:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227736.884215:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720712256:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227736.884499:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227736.884505:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227736.888455:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720712128:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227736.888482:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720712256:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227736.888506:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720712192:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227736.888957:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227736.888963:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227736.888968:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720712320:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227736.944298:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227737.456791:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227737.968071:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227738.480306:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227738.991958:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227739.506786:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227740.016130:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227740.528309:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227741.043198:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227741.554514:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761227742.000103:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720712384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227742.000306:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227742.000314:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720712448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227742.001071:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227742.001078:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227742.001082:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227742.001092:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3335 00000100:00080000:1.0:1761227742.001096:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227742.001102:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227742.001115:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227742.001134:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001c661ef2 x1846777720712576/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227742.001154:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227742.001158:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227742.001179:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227742.001191:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720712576:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227742.001192:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720712512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227742.005189:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720712512:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227742.008249:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720712384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227742.008750:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227742.008756:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227742.008760:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720712576:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227742.019254:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720712448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227742.064156:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227742.576212:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227743.091175:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227743.600810:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227744.117213:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227744.624679:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227745.136033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227745.649899:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227746.160038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227746.674051:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227747.121298:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227747.121305:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227747.121309:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227747.121321:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3340 00000100:00080000:1.0:1761227747.121325:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227747.121332:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227747.121347:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227747.121381:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720712832/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227747.121461:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720712832:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:1.0:1761227747.121576:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227747.121580:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227747.121698:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720712640:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227747.121730:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720712768:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227747.121757:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227747.121762:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720712704:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227747.132742:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720712704:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227747.132768:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720712640:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227747.132780:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720712768:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227747.133639:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227747.133646:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227747.133653:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720712832:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227747.184283:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227747.697390:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227748.208010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227748.720440:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227749.233785:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227749.751204:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227750.257056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227750.771258:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227751.280056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227751.793442:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227752.240906:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227752.240915:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227752.240919:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227752.240931:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3345 00000100:00080000:3.0:1761227752.240936:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227752.240943:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227752.240956:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227752.240992:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009d0a2521 x1846777720713088/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227752.241008:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227752.241012:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227752.241049:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720712960:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227752.241175:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761227752.241181:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720712896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227752.241207:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720713024:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227752.241647:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720713088:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227752.251428:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720712896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227752.251455:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720713024:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227752.251465:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720712960:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227752.254243:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227752.254256:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227752.254266:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720713088:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227752.304003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227752.818452:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227753.328391:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227753.841186:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227754.352309:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227754.865974:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227755.376017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227755.889209:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227756.400458:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227756.912426:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227757.360305:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227757.360315:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227757.360321:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227757.360335:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3350 00000100:00080000:0.0:1761227757.360340:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227757.360348:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:1.0:1761227757.360360:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720713152:192.168.202.114@tcp:400:kworker/u8:3.0 00000008:00100000:0.0:1761227757.360362:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227757.360388:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720713344/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761227757.360478:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720713280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227757.360497:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720713344:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227757.360521:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227757.360533:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720713216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227757.363139:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227757.363153:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227757.364569:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720713152:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227757.365189:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227757.365195:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227757.365295:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720713344:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227757.365479:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720713216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227757.365492:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720713280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227757.423989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227757.936184:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227758.447984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227758.964124:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227759.472348:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227759.984285:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227760.495995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227761.008155:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227761.519986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227762.032130:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227762.480208:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227762.480216:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227762.480220:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227762.480231:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3355 00000100:00080000:1.0:1761227762.480234:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227762.480240:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227762.480250:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227762.480265:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720713600/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227762.480292:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227762.480313:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227762.480379:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720713408:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227762.480477:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720713536:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227762.480510:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227762.480516:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720713472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227762.482547:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720713600:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227762.485663:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720713408:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227762.486575:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227762.486585:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227762.486593:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720713600:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227762.488143:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720713472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227762.488164:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720713536:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227762.548973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227763.058896:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227763.568007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227764.085367:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227764.592353:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227765.105916:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227765.616047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227766.132875:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227766.640628:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227767.152794:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227767.600570:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720713664:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227767.600749:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227767.600756:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720713728:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227767.602069:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227767.602076:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227767.602080:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227767.602103:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3360 00000100:00080000:0.0:1761227767.602106:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227767.602113:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227767.602127:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227767.602144:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720713856/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227767.602168:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227767.602171:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227767.602515:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227767.602533:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720713792:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227767.603184:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720713856:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227767.603748:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720713664:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227767.605189:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720713728:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227767.607353:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720713792:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227767.610641:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227767.610655:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227767.610665:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720713856:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227767.664082:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227768.183669:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227768.688034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227769.200204:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227769.712420:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227770.224138:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227770.736029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227771.257112:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227771.759999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227772.272163:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227772.720172:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720713920:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227772.720330:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227772.720337:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720713984:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227772.721722:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227772.721729:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227772.721733:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227772.721741:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3365 00000100:00080000:0.0:1761227772.721744:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227772.721750:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227772.721761:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227772.721777:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018849446 x1846777720714112/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761227772.721834:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720714112:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761227772.723110:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227772.723117:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227772.723153:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227772.723168:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720714048:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227772.723353:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720713984:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227772.730516:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720713920:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227772.730602:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720714048:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227772.732332:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227772.732342:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227772.732350:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720714112:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227772.784391:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227773.300136:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227773.809771:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227774.320727:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227774.833073:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227775.345099:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227775.855997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227776.368096:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227776.882869:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227777.392334:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227777.847162:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227777.847172:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227777.847176:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227777.847186:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3370 00000100:00100000:3.0:1761227777.847187:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720714176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227777.847190:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227777.847196:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227777.847205:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227777.847219:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006be78503 x1846777720714368/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227777.847248:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227777.847252:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227777.847273:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720714240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227777.847283:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720714304:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227777.847391:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720714368:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227777.861097:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720714304:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227777.861164:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720714240:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227777.861457:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227777.861464:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227777.861469:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720714368:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227777.861526:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720714176:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227777.904081:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227778.418942:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227778.928300:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227779.440071:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227779.951998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227780.464131:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227780.976663:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227781.490341:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227782.000031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227782.512843:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227782.960150:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227782.960160:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227782.960165:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227782.960178:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3376 00000100:00080000:3.0:1761227782.960183:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227782.960191:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227782.960206:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227782.960228:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092860b64 x1846777720714624/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227782.960282:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227782.960290:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227782.960670:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720714496:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227782.960800:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761227782.960807:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720714432:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227782.960835:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720714560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227782.961165:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720714624:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227782.975808:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720714432:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227782.975836:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720714560:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227782.975848:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720714496:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227782.976223:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227782.976229:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227782.976234:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720714624:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227783.024029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227783.536567:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227784.048036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227784.561786:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227785.072021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227785.584134:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227786.096003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227786.610363:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227787.120035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227787.633936:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227788.085120:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227788.085130:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227788.085134:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227788.085147:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3381 00000100:00080000:0.0:1761227788.085150:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227788.085157:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227788.085168:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227788.085184:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720714880/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227788.085210:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227788.085213:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227788.085314:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720714688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227788.085431:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720714816:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227788.085476:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227788.085482:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720714752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227788.088970:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720714880:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227788.099451:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720714752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227788.099483:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720714688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227788.099742:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720714816:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227788.099881:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227788.099887:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227788.099892:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720714880:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227788.145706:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227788.656055:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227789.169849:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227789.681649:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227790.192043:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227790.704346:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227791.215938:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227791.728186:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227792.240087:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227792.752173:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227793.202295:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720714944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227793.203822:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227793.203831:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227793.203836:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227793.203849:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3386 00000100:00100000:3.0:1761227793.203866:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715072:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227793.203875:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227793.203883:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227793.203896:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227793.203915:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046c2349c x1846777720715136/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227793.203940:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227793.203944:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227793.203945:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761227793.203971:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715008:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227793.205254:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720715136:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227793.212296:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720714944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227793.218221:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715008:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227793.218239:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715072:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227793.218876:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227793.218882:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227793.218888:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720715136:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227793.264556:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227793.777440:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227794.287984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227794.800920:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227795.312425:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227795.825822:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227796.337757:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227796.848140:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227797.360474:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227797.872159:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227798.320524:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227798.320533:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227798.320537:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227798.320550:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3391 00000100:00080000:3.0:1761227798.320554:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227798.320561:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227798.320573:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227798.320592:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8803538 x1846777720715392/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227798.320618:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227798.320621:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227798.320654:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715264:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227798.320786:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761227798.320793:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720715200:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227798.320823:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227798.320953:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720715392:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227798.327725:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715328:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227798.327747:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715264:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:02020000:1.0:1761227798.335892:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:1.0:1761227798.335906:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227798.335912:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227798.335916:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720715392:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227798.336092:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720715200:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227798.385137:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227798.896318:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227799.408911:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227799.920920:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227800.433703:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227800.944049:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227801.456050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227801.968087:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227802.480027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227802.992129:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227803.440793:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720715456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227803.441645:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761227803.441658:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227803.441670:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227803.441694:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227803.441699:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227803.441706:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3396 00000100:00080000:2.0:1761227803.441709:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227803.441716:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227803.441730:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227803.441751:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b3214ada x1846777720715648/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761227803.441815:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715584:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227803.441842:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720715648:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:2.0:1761227803.442988:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227803.442999:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227803.447885:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715584:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227803.447911:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720715520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227803.447921:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720715456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227803.449090:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227803.449105:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227803.449117:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720715648:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227803.503989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227804.016062:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227804.528011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227805.040061:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227805.552006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227806.064123:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227806.575995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227807.088405:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227807.599981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227808.112899:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227808.560112:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227808.560120:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227808.560124:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227808.560137:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3401 00000100:00080000:2.0:1761227808.560141:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227808.560149:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227808.560163:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227808.560184:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003ba27032 x1846777720715904/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227808.560214:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227808.560218:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227808.560318:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720715712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227808.560414:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720715840:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227808.560449:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227808.560454:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720715776:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227808.560632:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720715904:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227808.571725:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720715776:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227808.571747:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720715840:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227808.572349:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227808.572354:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227808.572359:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720715904:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227808.576521:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720715712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227808.624687:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227809.137684:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227809.647980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227810.160669:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227810.671983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227811.185038:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227811.696028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227812.208354:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227812.723196:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227813.232227:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227813.682974:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227813.682983:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227813.682987:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227813.683001:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3406 00000100:00080000:2.0:1761227813.683005:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227813.683013:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227813.683028:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227813.683050:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720716160/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227813.683073:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227813.683077:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227813.683118:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227813.683287:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720716160:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227813.683298:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761227813.683310:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720715968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227813.683352:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227813.704192:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227813.704213:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227813.704226:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720716160:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227813.704299:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720715968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227813.704326:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716096:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227813.704337:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227813.744111:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227814.256353:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227814.769754:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227815.280366:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227815.792047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227816.304048:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227816.815987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227817.328149:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227817.842478:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227818.355802:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227818.800575:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227818.800583:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227818.800587:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1761227818.800592:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720716224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227818.800597:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3411 00000100:00080000:3.0:1761227818.800602:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227818.800610:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227818.800631:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227818.800654:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c2618360 x1846777720716416/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227818.800682:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227818.800686:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227818.800706:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720716288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227818.800717:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716352:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227818.800821:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720716416:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:0.0:1761227818.806371:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227818.806385:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227818.806397:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720716416:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227818.806474:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720716224:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227818.806490:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716352:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227818.808488:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720716288:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227818.863974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227819.376068:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227819.887998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227820.401340:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227820.912029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227821.424825:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227821.936015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227822.448818:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227822.960028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227823.472921:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227823.920165:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227823.920174:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227823.920178:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227823.920192:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3417 00000100:00080000:2.0:1761227823.920197:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227823.920203:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227823.920226:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227823.920264:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072dbf25b x1846777720716672/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227823.920290:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227823.920294:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227823.920331:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716544:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227823.920404:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720716672:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227823.920452:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761227823.920461:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720716480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227823.920484:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716608:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227823.923692:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716544:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227823.930753:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227823.930764:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227823.930774:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720716672:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227823.931185:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720716608:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227823.931210:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720716480:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227823.985596:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227824.497351:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227825.009793:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227825.520914:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227826.031983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227826.544185:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227827.056002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227827.568896:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227828.080001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227828.592067:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227829.040164:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227829.040171:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227829.040175:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227829.040185:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3422 00000100:00080000:3.0:1761227829.040204:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227829.040211:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227829.040222:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227829.040238:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f64df33e x1846777720716928/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227829.040264:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227829.040267:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227829.040296:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720716800:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227829.040361:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720716928:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227829.040397:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761227829.040405:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720716736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227829.040447:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720716864:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227829.045975:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227829.045993:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227829.046003:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720716928:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227829.052101:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720716736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227829.052129:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720716864:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227829.052140:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720716800:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227829.103995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227829.620441:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227830.127989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227830.641050:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227831.152114:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227831.667484:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227832.176020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227832.688091:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227833.200023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227833.713437:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227834.161465:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227834.161493:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227834.161497:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227834.161506:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3427 00000100:00080000:3.0:1761227834.161510:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227834.161516:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227834.161525:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227834.161542:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a6d0bfc0 x1846777720717184/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227834.161568:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227834.161572:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227834.161598:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720717056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227834.161702:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761227834.161707:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720716992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227834.161727:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720717120:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227834.163773:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720717184:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227834.171142:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720716992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227834.171168:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720717120:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227834.171179:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720717056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227834.172200:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761227834.172210:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761227834.172218:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720717184:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227834.225044:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227834.736183:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227835.247994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227835.760036:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227836.272100:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227836.784136:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227837.296071:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227837.809173:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227838.320007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227838.832062:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227839.282673:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227839.282682:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227839.282687:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227839.282698:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3432 00000100:00080000:1.0:1761227839.282702:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227839.282708:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227839.282785:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227839.282805:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012902ca7 x1846777720717440/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227839.282836:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227839.282840:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227839.282962:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720717312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227839.282998:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720717248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227839.283169:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720717440:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761227839.283598:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720717376:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227839.286805:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720717312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227839.286815:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720717248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227839.286826:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720717376:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227839.287032:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227839.287037:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227839.287042:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720717440:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227839.343987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227839.858656:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227840.368551:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227840.880117:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227841.391990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227841.907443:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227842.415978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227842.929083:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227843.439977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227843.952824:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227844.400150:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720717504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227844.404983:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720717568:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227844.406504:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720717504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227844.409209:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720717632:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227844.410828:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227844.410837:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227844.410841:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227844.410851:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3437 00000100:00080000:3.0:1761227844.410878:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227844.410884:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227844.410897:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227844.410924:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e86c3a42 x1846777720717696/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227844.410942:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227844.410946:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227844.411231:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720717696:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227844.413945:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720717568:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227844.417049:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227844.417064:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227844.417075:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720717696:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227844.417505:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720717632:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227844.464795:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227844.977244:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227845.488547:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227846.000555:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227846.512017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227847.024197:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227847.536070:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227848.048381:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227848.559983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227849.072106:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227849.520185:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720717760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227849.520711:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761227849.520719:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720717824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227849.520893:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720717888:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227849.520975:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227849.520983:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227849.520987:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227849.520994:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3442 00000100:00080000:2.0:1761227849.520996:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227849.521003:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227849.521019:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227849.521039:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046bdcfa1 x1846777720717952/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227849.521055:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227849.521058:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227849.521072:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720717952:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227849.524281:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720717888:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227849.524304:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720717824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227849.525463:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720717760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227849.526809:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227849.526819:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227849.526828:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720717952:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227849.584007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227850.098073:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227850.607990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227851.122182:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227851.631962:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227852.145483:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227852.656037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227853.168192:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227853.679917:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227854.192232:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227854.640564:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227854.640572:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227854.640577:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227854.640589:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3447 00000100:00080000:2.0:1761227854.640593:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227854.640600:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227854.640613:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227854.640631:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001379e0d8 x1846777720718208/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227854.640651:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227854.640671:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227854.640715:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720718208:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227854.641250:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720718016:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227854.641307:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718144:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227854.641336:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227854.641343:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718080:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227854.648826:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718080:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227854.648851:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718144:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227854.651296:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720718016:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227854.655311:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227854.655325:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227854.655338:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720718208:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227854.707371:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227855.216352:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227855.729047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227856.242162:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227856.751966:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227857.264133:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227857.776895:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227858.288144:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227858.799993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227859.313815:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227859.760505:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720718272:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227859.761789:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227859.761795:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227859.761798:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227859.761805:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3452 00000100:00080000:3.0:1761227859.761808:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227859.761814:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227859.761823:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227859.761836:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000341ad5cd x1846777720718464/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227859.761913:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227859.762026:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227859.762032:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227859.762211:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720718464:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761227859.763441:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227859.763448:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227859.764476:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720718272:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227859.768520:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718400:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227859.768542:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718336:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227859.770066:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227859.770074:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227859.770082:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720718464:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227859.823986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227860.337961:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227860.847960:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227861.360081:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227861.872547:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227862.386828:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227862.896045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227863.409600:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227863.920641:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227864.432819:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227864.880573:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227864.880581:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227864.880585:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227864.880595:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3458 00000100:00080000:2.0:1761227864.880598:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227864.880604:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227864.880633:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227864.880650:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000790fc921 x1846777720718720/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227864.880677:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227864.880681:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227864.880712:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720718592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227864.880896:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720718720:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227864.881070:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720718528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227864.883390:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227864.889502:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227864.889514:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227864.889526:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720718720:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227864.889557:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720718528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227864.889585:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227864.889666:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720718592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227864.943996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227865.456176:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227865.968013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227866.481460:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227866.993082:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227867.504204:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227868.016003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227868.528162:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227869.040006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227869.555580:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227870.000311:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227870.000320:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227870.000324:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761227870.000329:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720718784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227870.000334:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3463 00000100:00080000:2.0:1761227870.000338:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227870.000345:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227870.000355:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227870.000374:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000003513afe x1846777720718976/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227870.000399:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227870.000403:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227870.000421:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227870.000424:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720718912:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227870.000459:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720718976:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227870.006931:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720718912:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227870.007849:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227870.007895:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227870.007904:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720718976:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227870.008550:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720718784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227870.008777:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720718848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227870.068158:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227870.577125:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227871.090506:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227871.600891:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227872.112062:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227872.624709:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227873.135961:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227873.648392:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227874.160038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227874.677026:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761227875.123657:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720719040:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761227875.124377:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227875.124384:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227875.124389:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227875.124396:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3468 00000100:00080000:0.0:1761227875.124400:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227875.124407:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227875.124421:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227875.124451:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720719232/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227875.124513:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720719232:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761227875.125592:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227875.125602:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227875.126673:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720719104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227875.126791:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227875.126801:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720719168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227875.138151:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227875.138167:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227875.138181:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720719232:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227875.142121:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720719168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227875.142174:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720719104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227875.142304:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720719040:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227875.184070:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227875.699801:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227876.208023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227876.720252:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227877.232159:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227877.746703:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227878.256021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227878.770098:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227879.279988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227879.792597:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227880.240962:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227880.240971:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227880.240975:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227880.240990:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3473 00000100:00080000:2.0:1761227880.240994:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227880.241002:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227880.241018:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227880.241039:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007221726f x1846777720719488/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227880.241068:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227880.241072:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227880.241215:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720719488:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227880.243310:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720719296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227880.243435:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227880.243469:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227880.243476:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719360:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227880.251138:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227880.251150:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227880.251160:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720719488:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227880.257391:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719360:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227880.257408:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720719296:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227880.257416:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719424:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227880.303994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227880.816113:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227881.328040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227881.842244:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227882.352310:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227882.864275:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227883.375995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227883.888873:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227884.400065:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227884.912248:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227885.361211:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720719552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227885.363484:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227885.363493:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227885.363497:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227885.363510:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3478 00000100:00080000:2.0:1761227885.363514:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227885.363530:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227885.363545:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227885.363580:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006bca6691 x1846777720719744/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227885.363608:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227885.363612:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227885.363633:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720719680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227885.363676:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227885.363904:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720719744:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227885.368578:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720719552:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227885.371971:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720719680:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227885.375347:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227885.375364:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227885.375378:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720719744:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227885.375499:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719616:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227885.424029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227885.940708:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227886.450238:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227886.963587:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227887.472763:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227887.984174:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227888.496051:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227889.008106:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227889.520026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227890.032149:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227890.480640:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227890.480650:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227890.480654:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227890.480669:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3483 00000100:00080000:2.0:1761227890.480673:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227890.480680:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227890.480694:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227890.480713:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720720000/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227890.480742:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227890.480745:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227890.480879:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720720000:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227890.481683:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720719808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227890.481759:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719936:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227890.481791:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227890.481798:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227890.487417:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227890.487434:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227890.487448:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720720000:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227890.487541:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719872:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227890.487554:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720719808:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227890.487566:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720719936:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227890.550177:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227891.056710:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227891.568042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227892.080302:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227892.592198:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227893.104332:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227893.616893:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227894.128157:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227894.642572:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227895.152075:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227895.600108:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227895.600115:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227895.600119:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227895.600133:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3488 00000100:00080000:1.0:1761227895.600137:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761227895.600144:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720720064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227895.600146:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227895.600160:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227895.600179:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720720256/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761227895.600243:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720720256:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227895.600254:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720720192:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227895.600296:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761227895.600301:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720720128:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227895.602058:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227895.602071:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761227895.606220:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227895.606234:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227895.606245:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720720256:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227895.608370:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720720128:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227895.608392:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720720064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227895.608405:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720720192:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227895.664026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227896.176463:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227896.688017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227897.200210:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227897.714474:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227898.224772:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227898.736020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227899.248197:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227899.760310:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227900.272579:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227900.720837:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227900.720844:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227900.720848:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227900.720900:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3493 00000100:00080000:3.0:1761227900.720904:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227900.720911:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227900.720924:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227900.720942:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b01aecf x1846777720720512/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227900.720967:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227900.720970:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227900.721011:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720720384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227900.721114:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761227900.721119:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720720320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227900.721137:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720720448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227900.721478:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720720512:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227900.730892:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720720320:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227900.730925:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720720448:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227900.730937:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720720384:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227900.734450:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227900.734465:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227900.734477:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720720512:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227900.784039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227901.296240:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227901.809208:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227902.320970:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227902.833204:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227903.344528:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227903.856082:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227904.368072:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227904.880175:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227905.395622:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227905.840217:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227905.840227:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227905.840232:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227905.840248:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3498 00000100:00080000:2.0:1761227905.840252:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227905.840260:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227905.840291:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227905.840312:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec5c4aea x1846777720720768/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761227905.840416:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720720768:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227905.840643:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720720576:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227905.840689:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720720704:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227905.840722:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227905.840728:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720720640:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227905.841280:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227905.841287:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761227905.847441:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227905.847472:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227905.847484:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720720768:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227905.847575:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720720640:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227905.847589:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720720704:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227905.854390:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720720576:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227905.910470:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227906.416084:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227906.928003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227907.440497:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227907.952167:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227908.464996:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227908.978516:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227909.488284:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227910.000019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227910.512421:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227910.962271:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720720832:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227910.965690:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761227910.965714:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720720896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227910.969944:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720720832:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227910.970394:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227910.970400:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227910.970405:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227910.970413:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3504 00000100:00100000:3.0:1761227910.970414:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720720960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227910.970416:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227910.970427:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227910.970441:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227910.970469:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720721024/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227910.970492:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227910.970497:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227910.970590:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720721024:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227910.974421:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720720896:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227910.977139:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227910.977153:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227910.977163:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720721024:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227910.977520:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720720960:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227911.024014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227911.539429:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227912.049301:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227912.560025:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227913.072066:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227913.584139:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227914.096288:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227914.608802:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227915.121444:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227915.632130:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227916.083353:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227916.083361:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227916.083385:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227916.083400:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3509 00000100:00080000:3.0:1761227916.083404:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227916.083412:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227916.083426:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227916.083447:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035883416 x1846777720721280/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227916.083475:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227916.083479:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227916.083522:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720721152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227916.085109:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720721088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227916.085516:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720721280:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227916.085648:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720721216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227916.095140:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227916.095156:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227916.095167:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720721280:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227916.095304:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720721152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227916.095453:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720721088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227916.095469:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720721216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227916.144097:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227916.656988:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227917.168061:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227917.681163:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227918.192758:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227918.708084:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227919.216097:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227919.728191:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227920.240309:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227920.753821:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227921.200284:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227921.200293:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227921.200299:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227921.200316:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3514 00000100:00080000:0.0:1761227921.200320:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227921.200329:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227921.200344:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227921.200365:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000805a30b x1846777720721536/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227921.200388:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227921.200409:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227921.200455:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720721408:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227921.200561:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761227921.200625:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720721344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227921.200653:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720721472:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227921.203946:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720721536:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227921.206222:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720721344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227921.207251:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720721472:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227921.207269:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720721408:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227921.213389:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761227921.213405:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227921.214704:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720721536:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227921.265845:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227921.784069:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761227922.288991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227922.802504:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227923.316494:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227923.827513:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227924.336785:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227924.852118:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227925.359976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227925.872777:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227926.320095:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227926.320104:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227926.320108:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227926.320119:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3519 00000100:00080000:1.0:1761227926.320122:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227926.320129:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227926.320156:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227926.320171:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720721792/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227926.320191:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227926.320194:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227926.320218:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720721664:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227926.320299:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761227926.320304:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720721600:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227926.320323:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720721728:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227926.324108:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720721792:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227926.336756:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720721600:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227926.350012:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720721728:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227926.350031:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720721664:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227926.353914:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227926.353927:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227926.353936:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720721792:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227926.384760:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227926.901002:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227927.407993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227927.922250:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227928.432136:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227928.947213:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227929.455984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227929.968106:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227930.480012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227930.997589:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227931.440141:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227931.440150:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227931.440154:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227931.440164:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3524 00000100:00080000:3.0:1761227931.440168:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227931.440174:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227931.440183:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227931.440199:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003d27cd56 x1846777720722048/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761227931.440257:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720722048:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00080000:3.0:1761227931.440970:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227931.440974:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227931.441103:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720721856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227931.441169:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720721984:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227931.441197:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761227931.441203:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720721920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227931.476347:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227931.476360:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227931.476370:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720722048:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227931.476635:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720721920:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227931.476647:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720721856:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227931.476656:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720721984:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227931.504264:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227932.016012:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227932.528643:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227933.040090:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227933.552055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227934.065628:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227934.576045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227935.091325:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227935.600032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227936.113998:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227936.562146:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227936.562156:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227936.562182:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227936.562198:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3529 00000100:00080000:1.0:1761227936.562202:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227936.562210:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227936.562226:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227936.562249:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057d87282 x1846777720722304/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227936.562292:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227936.562296:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227936.562338:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720722176:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227936.562341:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720722304:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227936.562481:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720722112:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227936.562731:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722240:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227936.566156:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720722112:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227936.567561:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761227936.567562:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722240:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227936.567572:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227936.567579:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720722304:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227936.569051:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720722176:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227936.625394:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227937.136231:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227937.648590:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227938.160064:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227938.672141:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227939.185583:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227939.696803:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227940.208171:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227940.719991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227941.232190:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227941.680068:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227941.680076:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227941.680080:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227941.680091:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3534 00000100:00080000:0.0:1761227941.680094:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227941.680102:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227941.680113:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227941.680130:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000805a30b x1846777720722560/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227941.680154:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227941.680157:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227941.680180:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722432:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227941.680287:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761227941.680294:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720722368:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227941.680317:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722496:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227941.681593:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720722560:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227941.687901:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720722368:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227941.698554:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722496:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227941.698574:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722432:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227941.698679:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227941.698703:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227941.698708:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720722560:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227941.744528:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227942.256941:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227942.767989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227943.280711:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227943.792009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227944.305239:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227944.815998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227945.330944:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227945.840111:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227946.353090:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227946.800093:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227946.800103:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227946.800108:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227946.800121:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3539 00000100:00080000:0.0:1761227946.800125:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227946.800131:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227946.800143:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227946.800160:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720722816/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761227946.800187:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227946.800191:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227946.802173:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720722816:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227946.802414:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720722624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227946.802451:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227946.802477:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761227946.802503:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227946.810493:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227946.810517:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720722624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227946.810527:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720722752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227946.814002:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227946.814016:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227946.814026:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720722816:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227946.864004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227947.376060:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227947.892080:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227948.401364:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227948.911986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227949.424620:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227949.936006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227950.448091:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227950.959995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227951.478430:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227951.925127:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227951.925135:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227951.925139:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227951.925150:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3545 00000100:00080000:2.0:1761227951.925154:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227951.925161:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227951.925174:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227951.925204:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046c2349c x1846777720723072/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227951.925230:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227951.925234:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227951.925264:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720722944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227951.925372:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761227951.925378:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720722880:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227951.925401:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720723008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227951.926030:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720723072:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227951.929522:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720722880:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227951.929554:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720723008:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227951.929565:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720722944:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:02020000:1.0:1761227951.932736:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:1.0:1761227951.932749:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227951.932754:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227951.932759:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720723072:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227951.983997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227952.496877:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227953.008009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227953.525137:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227954.031994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227954.544797:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227955.056410:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227955.568060:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227956.082683:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227956.595234:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227957.040410:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227957.040419:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227957.040424:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227957.040437:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3550 00000100:00080000:2.0:1761227957.040441:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227957.040449:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227957.040462:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227957.040489:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720723328/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761227957.040513:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227957.040517:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227957.040612:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720723136:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227957.040709:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720723264:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227957.040741:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761227957.040746:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720723200:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227957.041058:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720723328:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227957.046415:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720723200:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227957.046432:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720723264:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227957.047562:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227957.047570:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227957.047576:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720723328:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227957.048427:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720723136:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227957.107106:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227957.616121:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227958.128017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227958.640576:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227959.152202:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227959.667063:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227960.177264:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227960.688825:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227961.200189:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227961.712962:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227962.160959:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227962.160965:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227962.160969:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227962.160979:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3555 00000100:00080000:3.0:1761227962.160982:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227962.160996:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227962.161007:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227962.161022:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000006d85789 x1846777720723584/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227962.161039:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227962.161042:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227962.161119:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720723392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227962.161214:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720723520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227962.161247:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227962.161252:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720723456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227962.161282:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720723584:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227962.165963:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720723456:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227962.165980:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720723392:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227962.165991:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720723520:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227962.166806:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227962.166813:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227962.166819:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720723584:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227962.224099:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227962.736190:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227963.248005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227963.760517:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227964.272830:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227964.784527:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227965.295947:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227965.808092:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227966.319999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227966.832077:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761227967.280095:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227967.280104:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227967.280109:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227967.280122:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3560 00000100:00080000:1.0:1761227967.280127:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761227967.280129:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720723648:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227967.280137:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227967.280156:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227967.280175:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000aa6e6fe7 x1846777720723840/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227967.280200:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227967.280204:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227967.280225:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720723712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227967.280234:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720723776:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227967.280524:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720723840:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227967.287103:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720723648:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227967.293264:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227967.293277:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227967.293288:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720723840:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227967.293397:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720723712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227967.293429:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720723776:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227967.344062:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227967.856715:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227968.368010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:2.0:1761227968.880371:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227969.391999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227969.904089:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227970.416002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227970.928203:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227971.440706:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227971.952017:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761227972.400338:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227972.400348:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761227972.400352:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761227972.400367:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3565 00000100:00080000:2.0:1761227972.400371:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761227972.400380:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761227972.400396:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761227972.400418:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720724096/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761227972.400438:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720723904:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227972.400490:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720724096:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227972.400587:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720724032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227972.400637:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761227972.400652:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720723968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227972.406585:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720723904:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:2.0:1761227972.407458:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761227972.407468:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761227972.415626:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227972.415643:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227972.415674:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720724096:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761227972.415827:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720723968:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761227972.415841:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720724032:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227972.464056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227972.976230:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227973.488023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227974.000241:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227974.513615:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227975.024175:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227975.536061:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761227976.048535:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227976.560019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227977.075170:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761227977.520741:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720724160:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227977.522008:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761227977.522023:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720724224:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227977.522434:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227977.522439:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761227977.522443:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761227977.522449:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3570 00000100:00080000:1.0:1761227977.522452:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761227977.522460:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761227977.522474:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761227977.522493:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007ef613dc x1846777720724352/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761227977.522519:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761227977.522523:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761227977.522537:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761227977.522545:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720724288:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227977.522608:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720724352:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761227977.526122:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720724288:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227977.526932:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227977.526939:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227977.526945:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720724352:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761227977.527080:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720724224:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761227977.527092:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720724160:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227977.584708:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227978.096135:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227978.608045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227979.120163:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227979.632343:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227980.144117:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227980.656261:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227981.168829:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227981.680429:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227982.192459:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227982.640772:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227982.640780:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227982.640784:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227982.640797:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3575 00000100:00080000:3.0:1761227982.640801:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227982.640809:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227982.640842:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227982.640895:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720724608/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227982.640923:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227982.640927:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227982.640980:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720724416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227982.641092:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720724544:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227982.641130:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761227982.641136:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720724480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761227982.641837:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720724608:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761227982.648235:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761227982.648244:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720724480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761227982.648247:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761227982.648252:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720724608:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227982.648260:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720724416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761227982.648271:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720724544:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227982.704042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227983.217042:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227983.728071:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227984.240110:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227984.754331:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227985.264521:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227985.776066:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227986.288389:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227986.800081:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227987.312958:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227987.760170:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227987.760180:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227987.760185:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227987.760201:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3580 00000100:00080000:3.0:1761227987.760205:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227987.760214:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227987.760249:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227987.760273:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720724864/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761227987.760363:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720724864:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761227987.761301:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720724672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761227987.761362:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227987.761367:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761227987.761388:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720724736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227987.762516:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720724800:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761227987.768443:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227987.768474:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227987.768486:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720724864:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761227987.768625:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720724736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227987.768736:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720724672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227987.768752:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720724800:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761227987.824281:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227988.336192:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227988.848028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227989.367155:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227989.872057:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227990.384122:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227990.897420:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227991.411360:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227991.920520:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227992.437415:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761227992.880175:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227992.880184:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761227992.880188:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761227992.880202:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3586 00000100:00080000:3.0:1761227992.880205:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761227992.880214:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761227992.880228:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761227992.880247:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000088958979 x1846777720725120/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761227992.880272:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761227992.880276:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761227992.880388:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720724928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227992.880510:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720725056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227992.880550:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761227992.880559:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720724992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227992.880698:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720725120:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227992.884172:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720725056:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227992.884208:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720724992:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761227992.888195:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761227992.888208:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761227992.888220:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720725120:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761227992.889728:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720724928:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227992.944104:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227993.457589:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227993.967996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227994.480158:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227994.994764:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227995.506262:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227996.016064:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227996.528070:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227997.039931:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227997.552125:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761227998.000132:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227998.000141:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761227998.000168:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761227998.000180:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3591 00000100:00080000:0.0:1761227998.000183:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761227998.000190:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761227998.000200:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761227998.000218:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000085a527fc x1846777720725376/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761227998.000282:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720725376:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227998.000303:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720725184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227998.000378:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720725312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761227998.000704:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761227998.000708:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761227998.000830:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720725248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761227998.009896:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720725248:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761227998.010028:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761227998.010035:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761227998.010041:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720725376:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761227998.010338:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720725184:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761227998.010593:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720725312:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761227998.063954:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761227998.577350:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761227999.090186:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761227999.600574:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228000.113911:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228000.625146:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228001.135987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228001.648121:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228002.161217:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228002.672220:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228003.127393:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228003.127401:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228003.127404:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228003.127415:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3596 00000100:00080000:3.0:1761228003.127418:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228003.127424:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228003.127434:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228003.127447:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720725632/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228003.127473:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228003.127476:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228003.127798:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720725440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228003.127956:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720725568:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228003.127996:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228003.128002:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720725504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228003.128122:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720725632:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228003.139348:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720725568:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228003.139402:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720725440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228003.139823:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720725504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228003.140230:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228003.140235:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228003.140239:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720725632:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228003.184015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228003.697503:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228004.208007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228004.720081:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228005.232404:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228005.744152:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228006.256082:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228006.768119:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228007.279999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228007.792087:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228008.240129:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228008.240139:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228008.240164:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1761228008.240172:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720725696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761228008.240175:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3601 00000100:00080000:3.0:1761228008.240181:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228008.240189:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228008.240200:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228008.240219:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd946368 x1846777720725888/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228008.240249:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228008.240253:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228008.240294:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720725760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761228008.240305:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720725824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761228008.242489:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720725888:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:2.0:1761228008.247382:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720725696:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761228008.249173:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720725824:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:1.0:1761228008.249204:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228008.249210:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228008.249215:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720725888:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:3.0:1761228008.249312:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720725760:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:2.0:1761228008.304642:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228008.816569:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228009.327982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228009.842950:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228010.352787:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228010.866282:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228011.376261:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228011.888574:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228012.400002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228012.912122:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228013.367292:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720725952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228013.368687:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228013.368700:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228013.368704:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228013.368713:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3606 00000100:00080000:1.0:1761228013.368717:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228013.368724:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228013.368737:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228013.368754:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720726144/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228013.368806:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720726016:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228013.368851:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720726144:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228013.369288:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228013.369300:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720726080:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228013.370349:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228013.370358:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228013.375147:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720726080:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228013.375163:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720726016:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228013.375767:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720725952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228013.381131:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228013.381146:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228013.381159:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720726144:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228013.424510:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228013.936078:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228014.448078:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228014.962643:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228015.472076:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228015.984571:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228016.496027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228017.010849:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228017.520101:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228018.032989:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228018.480111:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228018.480119:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228018.480123:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228018.480135:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3611 00000100:00080000:1.0:1761228018.480156:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228018.480163:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228018.480177:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228018.480197:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720726400/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228018.480221:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228018.480224:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228018.480254:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720726272:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228018.480358:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761228018.480364:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720726208:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228018.480388:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720726336:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228018.482352:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720726400:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228018.491186:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720726336:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228018.491209:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720726272:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228018.492329:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228018.492339:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228018.492348:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720726400:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228018.493218:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720726208:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228018.544045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228019.057740:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228019.568904:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228020.081174:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228020.592771:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228021.104076:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228021.616058:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228022.128145:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228022.640901:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228023.152848:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228023.601497:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228023.601507:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228023.601510:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228023.601520:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3616 00000100:00080000:1.0:1761228023.601524:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228023.601550:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228023.601561:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228023.601580:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720726656/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228023.601607:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228023.601612:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228023.601657:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720726656:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228023.603098:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720726464:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228023.603216:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720726592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228023.603253:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228023.603260:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720726528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228023.606526:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228023.606539:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228023.606721:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720726656:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228023.609095:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720726528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228023.609113:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720726464:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228023.609123:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720726592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228023.663978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228024.176084:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228024.687996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228025.203324:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228025.712012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228026.225740:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228026.736700:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228027.250510:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228027.764930:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228028.278665:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228028.720353:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228028.720360:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228028.720364:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228028.720374:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3621 00000100:00080000:3.0:1761228028.720377:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228028.720382:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228028.720392:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228028.720407:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008dec2428 x1846777720726912/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228028.720434:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228028.720456:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228028.720486:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720726720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228028.720568:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720726848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228028.720593:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228028.720597:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720726784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228028.726893:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720726912:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761228028.730656:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228028.730667:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228028.730677:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720726912:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228028.730772:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720726784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228028.730796:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720726720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228028.730809:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720726848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228028.783994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228029.296942:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228029.807968:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228030.320738:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228030.833126:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228031.344286:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228031.857420:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228032.372245:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228032.881961:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228033.394221:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228033.844953:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228033.844961:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228033.844965:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228033.844987:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3626 00000100:00080000:1.0:1761228033.844990:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228033.844996:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228033.845015:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228033.845028:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720727040:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228033.845031:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a2fc13a x1846777720727168/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228033.845065:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228033.845079:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228033.845098:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720726976:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228033.845169:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720727104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228033.845223:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720727168:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228033.850616:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720727040:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228033.851846:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720727104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228033.851900:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720726976:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228033.852943:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228033.852951:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228033.852958:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720727168:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228033.903975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228034.425262:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228034.929179:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228035.440541:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228035.952053:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228036.464183:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228036.977074:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228037.488046:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228037.999979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228038.512219:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228038.972168:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228038.972178:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228038.972182:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228038.972194:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3632 00000100:00080000:3.0:1761228038.972198:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228038.972225:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228038.972235:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228038.972252:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e86c3a42 x1846777720727424/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228038.972282:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228038.972285:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228038.972317:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720727296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228038.973624:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761228038.973644:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720727232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228038.974500:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720727360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228038.977491:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720727424:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228038.982063:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720727232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228038.982088:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720727360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228038.982099:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720727296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228038.982162:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228038.982167:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228038.982172:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720727424:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228039.023998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228039.536084:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228040.047996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228040.560090:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228041.072020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228041.584775:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228042.105455:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228042.608060:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228043.120005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228043.633212:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228044.080098:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228044.080107:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228044.080112:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228044.080125:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3637 00000100:00080000:0.0:1761228044.080130:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761228044.080132:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720727488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228044.080140:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228044.080154:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228044.080175:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018849446 x1846777720727680/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228044.080206:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228044.080211:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228044.080241:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720727616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228044.080256:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720727552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228044.080379:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720727680:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228044.084958:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720727552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228044.085073:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720727488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228044.085084:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720727616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228044.085600:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228044.085605:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228044.085611:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720727680:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228044.144887:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228044.656298:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228045.168295:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228045.680216:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228046.191973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228046.704651:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228047.215982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228047.731160:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228048.239996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228048.752069:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228049.201248:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228049.201256:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228049.201260:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228049.201271:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3642 00000100:00080000:0.0:1761228049.201275:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228049.201281:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228049.201309:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228049.201325:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000de75897f x1846777720727936/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228049.201352:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228049.201356:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228049.201491:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720727744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228049.201585:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720727872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228049.201618:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228049.201624:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720727808:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228049.201807:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720727936:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228049.208061:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720727808:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228049.208080:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761228049.208083:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720727744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228049.208087:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228049.208094:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720727936:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228049.208098:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720727872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228049.267745:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228049.779642:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228050.288023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228050.800033:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228051.311986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228051.824073:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228052.336669:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228052.848729:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228053.361099:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228053.872158:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228054.321113:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228054.321123:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228054.321127:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228054.321141:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3647 00000100:00080000:1.0:1761228054.321165:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228054.321172:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228054.321186:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228054.321208:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001c661ef2 x1846777720728192/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761228054.321298:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720728192:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228054.321530:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720728000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228054.321566:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720728128:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228054.321598:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228054.321604:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720728064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228054.321836:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228054.321841:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761228054.325932:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228054.325946:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228054.325958:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720728192:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228054.326068:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720728064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228054.326081:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720728000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228054.326092:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720728128:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228054.387505:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228054.896705:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228055.408224:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228055.920269:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228056.432391:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228056.944314:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228057.458580:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228057.968130:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228058.480015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228058.998034:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228059.441070:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228059.441078:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228059.441082:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228059.441096:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3652 00000100:00080000:3.0:1761228059.441100:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228059.441107:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228059.441121:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228059.441139:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f64df33e x1846777720728448/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228059.441171:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228059.441175:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228059.443251:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720728256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228059.443398:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720728384:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228059.443441:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228059.443449:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720728320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228059.445779:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720728448:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761228059.455166:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228059.455179:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228059.455189:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720728448:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228059.455323:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720728320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228059.455346:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720728256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228059.455357:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720728384:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228059.504590:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228060.025025:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228060.527973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228061.040224:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228061.554352:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228062.064138:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228062.576032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228063.088158:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228063.602371:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228064.112094:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228064.562610:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228064.562629:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228064.562633:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228064.562646:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3657 00000100:00080000:1.0:1761228064.562650:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228064.562657:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228064.562668:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228064.562684:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720728704/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228064.562710:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228064.562714:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228064.562849:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720728512:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228064.562980:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720728640:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228064.563014:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228064.563020:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720728576:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228064.563423:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720728704:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761228064.572016:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228064.572028:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228064.572038:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720728704:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228064.575652:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720728640:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228064.575688:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720728576:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228064.575697:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720728512:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228064.624003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228065.136623:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228065.648845:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228066.160357:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228066.673232:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228067.184157:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228067.696093:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228068.215816:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228068.720011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228069.232103:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228069.680102:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228069.680126:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228069.680130:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228069.680142:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3662 00000100:00080000:1.0:1761228069.680146:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228069.680187:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228069.680201:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228069.680223:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d0d44bb x1846777720728960/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228069.680248:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228069.680252:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228069.680351:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720728960:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228069.680932:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720728768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228069.681020:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720728896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228069.681058:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228069.681064:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720728832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228069.687925:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720728832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228069.687949:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720728768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228069.687959:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720728896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228069.689059:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228069.689070:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228069.689080:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720728960:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228069.744401:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228070.257042:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228070.768979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228071.280301:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228071.792241:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228072.304338:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228072.815995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228073.331424:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228073.839986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228074.352880:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228074.800164:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720729024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228074.801432:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228074.801437:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228074.801441:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228074.801448:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3667 00000100:00080000:2.0:1761228074.801451:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228074.801457:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228074.801467:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228074.801478:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072dbf25b x1846777720729216/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228074.801502:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228074.801505:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228074.801553:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720729088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228074.801616:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228074.801623:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720729152:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228074.802502:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720729216:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228074.804310:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720729024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228074.804540:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720729152:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228074.804555:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720729088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228074.804902:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228074.804907:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228074.804912:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720729216:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228074.863976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228075.377487:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228075.889031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228076.401144:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228076.912016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228077.424193:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228077.936984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228078.448177:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228078.959998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228079.474298:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228079.920157:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228079.920166:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228079.920171:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228079.920186:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3673 00000100:00080000:2.0:1761228079.920191:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228079.920200:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228079.920215:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228079.920234:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720729344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228079.920239:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a37cdda5 x1846777720729472/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228079.920273:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228079.920277:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228079.920333:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720729472:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228079.920349:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720729280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228079.928790:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720729408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228079.932567:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720729344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228079.933120:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228079.933126:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228079.933131:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720729472:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228079.933205:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720729280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228079.940171:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720729408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228079.983993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228080.496300:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228081.007983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228081.520133:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228082.031994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228082.544102:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228083.055990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228083.568835:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228084.079972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228084.594116:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228085.040126:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228085.040134:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228085.040155:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228085.040167:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3678 00000100:00080000:2.0:1761228085.040170:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228085.040176:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228085.040187:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228085.040203:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084e78951 x1846777720729728/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228085.040235:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228085.040239:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228085.040251:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720729536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228085.040276:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720729600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228085.040324:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720729728:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228085.040349:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720729664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228085.045598:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720729600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228085.050202:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720729536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228085.050230:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720729664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228085.050354:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228085.050360:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228085.050365:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720729728:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228085.104029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228085.616207:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228086.131601:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228086.640092:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228087.151990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228087.664955:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228088.176089:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228088.688235:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228089.199998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228089.712172:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228090.161203:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228090.161212:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228090.161216:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228090.161226:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3683 00000100:00100000:1.0:1761228090.161227:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720729792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228090.161232:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228090.161242:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228090.161256:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228090.161278:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720729984/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228090.161298:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228090.161302:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228090.161351:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720729920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228090.161355:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720729856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228090.162079:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720729984:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228090.166012:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720729920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228090.172592:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720729792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228090.174485:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228090.174496:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228090.174506:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720729984:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228090.178204:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720729856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228090.224324:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228090.736300:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228091.248042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228091.760367:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228092.272364:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228092.785227:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228093.296704:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228093.808419:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228094.322405:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228094.832190:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228095.282143:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228095.282152:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228095.282156:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228095.282169:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3688 00000100:00080000:2.0:1761228095.282173:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228095.282180:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228095.282195:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228095.282218:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c9f23bda x1846777720730240/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228095.282246:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228095.282250:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228095.282292:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720730112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228095.282407:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761228095.282415:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720730048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228095.282443:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720730176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228095.282636:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720730240:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228095.287964:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720730048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228095.287995:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720730112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228095.295291:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720730176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228095.315974:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228095.315991:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228095.316003:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720730240:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228095.344060:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228095.857029:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228096.368476:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228096.880143:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228097.392187:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228097.906363:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228098.416008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228098.928130:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228099.440437:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228099.952094:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228100.402438:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720730304:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228100.403243:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730368:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228100.403319:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228100.403325:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228100.403330:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228100.403336:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3693 00000100:00080000:0.0:1761228100.403339:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228100.403347:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228100.403359:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228100.403377:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fe9fa97 x1846777720730496/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228100.403402:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228100.403406:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228100.403426:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228100.403432:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730432:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228100.403732:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720730496:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228100.407491:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720730304:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228100.409117:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730368:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228100.409958:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730432:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228100.415812:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228100.415828:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228100.415964:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720730496:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228100.465540:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228100.976292:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228101.488602:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228102.000497:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228102.512383:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228103.024501:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228103.536066:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228104.048648:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228104.560289:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228105.080129:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228105.520298:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228105.520306:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228105.520310:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228105.520323:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3698 00000100:00080000:0.0:1761228105.520327:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228105.520334:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228105.520347:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228105.520383:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000243e8b76 x1846777720730752/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228105.520399:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228105.520403:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228105.520718:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720730752:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228105.522298:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720730560:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228105.522371:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228105.522400:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228105.522405:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:02020000:3.0:1761228105.527705:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:3.0:1761228105.527717:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228105.527722:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228105.527726:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720730752:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228105.529003:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228105.529400:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228105.529416:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720730560:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228105.584295:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228106.096301:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228106.608002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228107.120358:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228107.634778:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228108.144993:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228108.657071:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228109.168397:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228109.680006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228110.192190:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228110.640147:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228110.640156:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228110.640160:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228110.640173:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3703 00000100:00080000:0.0:1761228110.640177:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228110.640185:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228110.640199:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228110.640217:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720731008/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228110.640242:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228110.640246:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228110.640272:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730880:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228110.640302:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720731008:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228110.640402:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761228110.640409:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720730816:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228110.640813:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228110.653187:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730880:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228110.653232:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228110.653239:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228110.653240:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720730816:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228110.653248:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720731008:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228110.653256:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720730944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228110.704037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228111.216689:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228111.728016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228112.240143:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228112.754342:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228113.264296:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228113.779232:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228114.288255:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228114.804135:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228115.313308:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228115.760365:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228115.760372:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228115.760376:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228115.760400:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3708 00000100:00080000:1.0:1761228115.760403:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228115.760409:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228115.760418:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228115.760431:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720731264/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228115.760452:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228115.760455:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228115.760473:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720731136:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228115.760551:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761228115.760556:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720731072:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228115.760572:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720731200:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228115.760759:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720731264:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228115.763684:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720731136:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228115.763717:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720731072:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228115.763729:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720731200:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228115.766343:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228115.766353:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228115.766362:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720731264:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228115.824718:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228116.336069:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228116.848010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228117.360252:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228117.872115:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228118.384558:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228118.896742:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228119.408128:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228119.919985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228120.432103:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228120.880341:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228120.880350:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228120.880354:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228120.880367:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3714 00000100:00080000:0.0:1761228120.880371:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228120.880378:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228120.880393:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228120.880414:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720731520/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228120.880444:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228120.880448:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228120.880482:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720731392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228120.880612:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761228120.880634:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720731328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228120.880656:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720731456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228120.880754:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720731520:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228120.883599:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720731328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228120.887396:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228120.887411:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228120.887423:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720731520:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228120.887605:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720731456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228120.887621:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720731392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228120.944604:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228121.457849:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228121.968025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228122.480152:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228122.992427:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228123.505993:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228124.016914:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228124.528954:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228125.040293:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228125.552274:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228126.002084:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228126.002091:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228126.002096:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228126.002109:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3719 00000100:00080000:0.0:1761228126.002113:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228126.002121:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228126.002133:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228126.002150:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000525c9db8 x1846777720731776/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228126.002172:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228126.002176:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228126.002265:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720731584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228126.002362:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720731712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228126.002404:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228126.002409:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720731648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228126.002708:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720731776:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228126.012337:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720731648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228126.012356:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720731712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228126.012799:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228126.012804:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228126.012809:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720731776:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228126.013125:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720731584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228126.064060:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228126.576135:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228127.088032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228127.600243:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228128.112143:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228128.624118:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228129.135951:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228129.648086:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228130.160026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228130.672103:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228131.122623:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228131.122630:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228131.122634:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228131.122647:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3724 00000100:00080000:2.0:1761228131.122651:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228131.122659:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:3.0:1761228131.122670:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720731840:192.168.202.114@tcp:400:kworker/u8:3.0 00000008:00100000:2.0:1761228131.122671:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228131.122689:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001379e0d8 x1846777720732032/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228131.122716:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228131.122720:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228131.122779:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720731904:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228131.122796:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720731968:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228131.123321:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720732032:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228131.129370:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720731904:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228131.129680:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720731968:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228131.131352:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720731840:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228131.138304:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228131.138339:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228131.138353:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720732032:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228131.183993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228131.698030:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228132.207992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228132.720209:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228133.233048:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228133.744333:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228134.256163:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228134.768120:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228135.279928:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228135.792169:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228136.240512:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228136.240521:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228136.240526:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228136.240540:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3729 00000100:00080000:0.0:1761228136.240561:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228136.240569:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228136.240584:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228136.240606:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720732288/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761228136.240693:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720732288:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228136.242919:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720732096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228136.243018:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720732224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228136.243052:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228136.243060:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720732160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228136.243792:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228136.243799:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:3.0:1761228136.247299:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228136.247317:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228136.247330:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720732288:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228136.249760:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720732096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228136.249784:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720732224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228136.250656:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720732160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228136.304021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228136.816226:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228137.328131:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228137.840050:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228138.351984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228138.864617:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228139.375975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228139.888826:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228140.403485:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228140.912216:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228141.364142:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228141.364152:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228141.364157:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228141.364172:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3734 00000100:00080000:1.0:1761228141.364177:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228141.364185:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228141.364200:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228141.364214:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720732352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228141.364222:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017ee9697 x1846777720732544/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228141.364330:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720732480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228141.364368:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228141.364507:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720732416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228141.364607:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228141.364614:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228141.364940:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720732544:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761228141.376920:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228141.376935:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228141.376947:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720732544:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228141.377700:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720732416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228141.377720:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720732352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228141.377760:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720732480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228141.424038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228141.936043:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228142.447982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228142.960095:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228143.472024:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228143.984092:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228144.496206:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228145.010786:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228145.519992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228146.032317:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228146.480304:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228146.480313:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228146.480317:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228146.480329:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3739 00000100:00080000:2.0:1761228146.480335:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 6s 00000100:00080000:2.0:1761228146.480343:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228146.480357:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228146.480380:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000003513afe x1846777720732800/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228146.480404:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228146.480408:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228146.480488:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720732608:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228146.480603:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720732736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228146.480642:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228146.480646:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720732672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228146.481009:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720732800:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228146.486954:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720732672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228146.486983:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720732736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228146.489636:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720732608:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228146.489760:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228146.489766:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228146.489771:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720732800:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228146.544061:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228147.056339:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228147.567983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228148.084983:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228148.591997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228149.104674:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228149.616041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228150.128498:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228150.644557:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228151.152066:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228151.600229:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228151.600237:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228151.600241:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228151.600254:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3744 00000100:00080000:0.0:1761228151.600258:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1761228151.600264:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228151.600275:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228151.600290:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720733056/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228151.600330:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228151.600334:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228151.600365:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720733056:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228151.600408:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720732864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228151.600656:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720732992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228151.600692:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228151.600699:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720732928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228151.607612:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720732928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228151.607630:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720732864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228151.607638:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720732992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228151.608221:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228151.608227:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228151.608232:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720733056:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228151.667657:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228152.176534:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228152.687982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228153.205710:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228153.711967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228154.224522:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228154.735993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228155.248155:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228155.759986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228156.272052:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228156.720233:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228156.720241:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228156.720245:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228156.720257:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3749 00000100:00080000:1.0:1761228156.720261:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 16s 00000100:00080000:1.0:1761228156.720269:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228156.720281:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228156.720313:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000961c7ab0 x1846777720733312/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228156.720330:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228156.720334:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228156.720355:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720733184:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228156.720448:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761228156.720455:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720733120:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228156.720478:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720733248:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228156.720737:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720733312:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228156.724901:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720733120:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228156.724932:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720733248:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228156.724943:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720733184:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228156.725080:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228156.725085:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228156.725089:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720733312:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228156.784055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228157.297022:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228157.807986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228158.320197:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228158.834474:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228159.344650:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228159.855980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228160.368111:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228160.879988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228161.392134:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228161.842394:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228161.842402:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228161.842407:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228161.842418:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3754 00000100:00080000:2.0:1761228161.842422:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228161.842429:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228161.842439:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228161.842454:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c987228a x1846777720733568/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228161.842521:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720733568:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761228161.842808:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228161.842812:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228161.842952:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720733376:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228161.843678:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720733440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228161.844412:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720733504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228161.851550:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228161.851563:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228161.851573:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720733568:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228161.852386:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720733376:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228161.852396:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720733440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228161.854458:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720733504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228161.904087:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228162.416749:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228162.930667:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228163.440122:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228163.951987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228164.464057:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228164.976004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228165.488153:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228166.000201:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228166.515602:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228166.960093:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228166.960102:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228166.960106:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228166.960121:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3760 00000100:00080000:0.0:1761228166.960125:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228166.960132:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228166.960145:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228166.960160:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720733824/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228166.960183:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228166.960186:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228166.960211:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720733696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228166.960280:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720733824:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228166.960304:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761228166.960319:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720733632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228166.960358:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720733760:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228166.964750:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1761228166.964758:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720733632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228166.964763:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228166.964779:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720733824:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228166.967628:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720733760:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228166.967650:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720733696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228167.025084:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228167.536323:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228168.048641:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228168.564134:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228169.072026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228169.584644:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228170.096563:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228170.609257:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228171.120087:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228171.633114:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228172.080169:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228172.080178:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228172.080183:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228172.080196:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3765 00000100:00080000:1.0:1761228172.080199:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761228172.080206:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228172.080218:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228172.080235:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720734080/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228172.080260:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228172.080284:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228172.080315:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720733952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228172.080416:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761228172.080422:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720733888:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228172.080448:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720734016:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228172.080483:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720734080:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228172.086731:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720733888:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228172.093086:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228172.093103:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228172.093115:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720734080:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228172.095745:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720734016:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228172.095769:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720733952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228172.144502:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228172.656211:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228173.175199:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228173.680490:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228174.192109:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228174.704190:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228175.215999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228175.728086:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228176.240231:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228176.752564:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228177.200489:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228177.200498:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228177.200502:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228177.200516:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3770 00000100:00080000:2.0:1761228177.200520:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228177.200528:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228177.200541:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228177.200559:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000790fc921 x1846777720734336/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228177.200589:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228177.200593:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228177.200617:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720734144:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228177.200721:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720734272:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228177.200752:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228177.200756:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720734208:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228177.201224:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720734336:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228177.210203:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720734208:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228177.210230:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720734144:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228177.211830:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720734272:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228177.214077:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228177.214088:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228177.214097:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720734336:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228177.264540:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228177.777066:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228178.289448:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228178.803908:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228179.312039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228179.824357:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228180.336651:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228180.848151:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228181.359979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228181.874082:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228182.320346:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228182.320370:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228182.320374:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228182.320385:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3775 00000100:00080000:2.0:1761228182.320389:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228182.320395:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228182.320406:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228182.320421:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c9f23bda x1846777720734592/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228182.320448:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228182.320451:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228182.320551:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720734592:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228182.320886:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720734400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228182.320921:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720734528:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228182.320942:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228182.320948:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720734464:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228182.329356:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228182.329372:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228182.329382:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720734592:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228182.329877:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720734464:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228182.329891:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720734400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228182.329901:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720734528:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228182.384050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228182.899708:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228183.408474:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228183.923033:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228184.432447:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228184.948188:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228185.456210:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228185.968219:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228186.480009:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228186.992167:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228187.442071:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228187.442079:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228187.442083:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228187.442096:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3780 00000100:00080000:1.0:1761228187.442100:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761228187.442106:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228187.442117:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228187.442132:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057d87282 x1846777720734848/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228187.442160:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228187.442163:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228187.442208:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720734848:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228187.442263:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720734656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228187.442303:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720734784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228187.442329:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228187.442335:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720734720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228187.449890:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720734656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228187.450027:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228187.450032:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228187.450051:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720734848:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228187.451493:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720734720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228187.451511:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720734784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228187.504479:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228188.016568:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228188.528034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228189.043886:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228189.551984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228190.064073:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228190.576004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228191.088573:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228191.600940:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228192.115381:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228192.560116:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228192.560126:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228192.560130:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228192.560141:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3785 00000100:00080000:2.0:1761228192.560145:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228192.560151:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228192.560161:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228192.560176:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003ba27032 x1846777720735104/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228192.560204:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228192.560208:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228192.560293:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720735104:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228192.563963:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720734912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228192.564095:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720735040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228192.564131:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228192.564138:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720734976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228192.568905:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228192.568917:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228192.568927:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720735104:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228192.571310:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720734912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228192.571332:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720735040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228192.578950:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720734976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228192.624555:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228193.136033:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228193.649453:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228194.160507:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228194.671999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228195.184638:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228195.695986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228196.208090:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228196.720248:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228197.233122:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228197.680068:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228197.680076:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228197.680096:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228197.680108:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3790 00000100:00080000:2.0:1761228197.680111:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228197.680118:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228197.680129:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228197.680143:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720735360/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228197.680169:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228197.680172:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228197.680218:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720735360:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228197.680260:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720735168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228197.680317:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720735296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228197.680345:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228197.680352:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720735232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228197.685426:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228197.685439:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228197.685450:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720735360:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228197.693990:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720735232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228197.694010:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720735168:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228197.694021:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720735296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228197.744003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228198.259709:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228198.767973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228199.281957:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228199.792013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228200.304098:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228200.816018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228201.328767:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228201.840094:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228202.352964:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228202.800055:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228202.800063:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228202.800067:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228202.800078:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3795 00000100:00080000:2.0:1761228202.800081:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228202.800089:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228202.800103:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228202.800119:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720735616/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228202.800145:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228202.800148:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228202.800229:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720735424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228202.800322:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720735552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228202.800352:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228202.800357:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720735488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228202.800654:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720735616:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228202.806655:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720735424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228202.810941:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720735552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228202.811360:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720735488:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228202.813851:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228202.813903:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228202.813914:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720735616:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228202.865480:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228203.376242:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228203.888132:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228204.400117:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228204.912458:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228205.424711:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228205.937126:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228206.448307:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228206.960494:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228207.472167:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228207.920170:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720735680:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228207.924031:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720735680:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228207.926466:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720735744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228207.926679:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228207.926708:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720735808:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228207.927132:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228207.927140:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228207.927144:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228207.927150:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3801 00000100:00080000:1.0:1761228207.927161:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761228207.927170:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228207.927187:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228207.927209:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720735872/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228207.927233:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228207.927236:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228207.927348:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720735872:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228207.937394:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720735808:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228207.937421:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720735744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228207.942582:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228207.942600:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228207.942613:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720735872:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228207.984033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228208.496450:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228209.008032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228209.520145:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228210.032222:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228210.544328:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228211.056047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228211.569114:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228212.081128:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228212.596337:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228213.045097:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228213.045106:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228213.045110:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228213.045123:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3806 00000100:00080000:2.0:1761228213.045126:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228213.045134:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228213.045148:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228213.045166:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f5134a53 x1846777720736128/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228213.045191:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228213.045195:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228213.045244:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720736128:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228213.045273:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720736000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228213.045405:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761228213.045412:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720735936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228213.045440:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720736064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228213.049911:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720736000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228213.053090:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720736064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228213.053274:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228213.053281:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228213.053286:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720736128:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228213.053350:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720735936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228213.105915:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228213.616072:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228214.128004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228214.640897:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228215.153392:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228215.664500:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228216.177215:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228216.690182:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228217.200363:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228217.712590:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228218.160127:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228218.160159:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720736192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228218.160449:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228218.160456:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228218.160462:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228218.160469:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3811 00000100:00080000:2.0:1761228218.160474:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228218.160499:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228218.160511:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228218.160529:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f5134a53 x1846777720736384/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228218.160555:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228218.160559:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228218.160577:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720736320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228218.160628:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228218.160634:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720736256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228218.162426:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720736384:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228218.164422:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720736192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228218.164567:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228218.164571:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228218.164576:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720736384:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228218.164883:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720736256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228218.164897:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720736320:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228218.224092:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228218.738111:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228219.252918:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228219.760232:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228220.272162:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228220.784165:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228221.295985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228221.808986:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228222.319952:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228222.836086:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228223.280130:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720736448:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228223.281315:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720736512:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228223.282179:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228223.282184:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228223.282188:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228223.282194:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3816 00000100:00080000:1.0:1761228223.282212:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761228223.282217:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228223.282227:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228223.282243:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001c661ef2 x1846777720736640/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228223.282273:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228223.282276:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228223.282303:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228223.282311:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720736576:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228223.282547:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720736640:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228223.286499:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720736576:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228223.286520:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720736512:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228223.286810:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720736448:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228223.287118:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228223.287123:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228223.287128:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720736640:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228223.343982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228223.856097:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228224.368012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228224.880106:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228225.391933:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228225.904053:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228226.434120:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228226.928769:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228227.440630:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228227.952061:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228228.401727:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228228.401734:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228228.401738:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228228.401749:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3821 00000100:00080000:0.0:1761228228.401752:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228228.401759:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228228.401769:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228228.401781:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092ccaebf x1846777720736896/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228228.401801:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228228.401804:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228228.401829:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720736768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228228.401931:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761228228.401937:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720736704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228228.401956:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720736832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228228.402070:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720736896:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228228.406784:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720736704:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228228.406802:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720736832:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228228.406811:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720736768:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228228.406891:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228228.406896:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228228.406900:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720736896:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228228.463983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228228.976096:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228229.488152:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228230.000077:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228230.511986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228231.024049:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228231.537277:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228232.048105:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228232.559974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228233.073930:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228233.521151:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228233.521177:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228233.521181:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228233.521193:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3826 00000100:00080000:2.0:1761228233.521196:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228233.521203:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228233.521214:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228233.521260:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008451bff4 x1846777720737152/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228233.521295:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228233.521298:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228233.521327:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720737024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228233.521431:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761228233.521438:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720736960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228233.521460:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720737088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228233.521548:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720737152:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761228233.527930:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228233.527947:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228233.527958:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720737152:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228233.531380:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720736960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228233.531409:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720737088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228233.531421:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720737024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228233.584975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228234.096133:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228234.608034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228235.120221:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228235.632163:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228236.145482:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228236.661421:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228237.167997:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228237.680109:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228238.192999:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228238.642019:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228238.642027:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228238.642031:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228238.642043:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3831 00000100:00080000:2.0:1761228238.642065:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228238.642073:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228238.642086:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228238.642104:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720737408/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228238.642142:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228238.642146:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228238.642187:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720737280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228238.642529:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720737216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228238.644262:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720737408:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228238.652090:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720737344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228238.652549:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720737280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228238.654006:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720737216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228238.659591:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228238.659606:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228238.659618:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720737408:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228238.660311:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720737344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228238.704013:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228239.218184:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228239.728027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228240.240790:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228240.688971:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720737472:192.168.202.114@tcp:17:kworker/1:2.0 00000100:00100000:1.0:1761228240.689068:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228240.689075:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720737536:192.168.202.114@tcp:17:kworker/1:2.0 00000100:00100000:0.0:1761228240.693971:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720737536:192.168.202.114@tcp:17:kworker/1:2.0 00000100:00100000:0.0:1761228240.693996:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720737472:192.168.202.114@tcp:17:kworker/1:2.0 00000100:00100000:3.0:1761228240.751998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228241.266157:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228241.776050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228242.288941:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228242.800379:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228243.312186:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228243.760181:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228243.760189:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228243.760202:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228243.760213:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3836 00000100:00080000:0.0:1761228243.760217:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228243.760224:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228243.760236:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228243.760252:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720737792/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228243.760273:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228243.760282:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228243.760308:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720737664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228243.760345:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720737792:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228243.760537:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761228243.760546:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720737600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228243.760588:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720737728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228243.766202:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228243.766216:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228243.766227:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720737792:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228243.766569:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720737600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228243.766586:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720737728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228243.766597:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720737664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228243.824031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228244.337202:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228244.848227:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228245.361597:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228245.875794:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228246.385280:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228246.896034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228247.408274:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228247.920267:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228248.434391:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228248.881009:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228248.881026:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228248.881031:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228248.881044:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3842 00000100:00080000:2.0:1761228248.881055:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228248.881079:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228248.881093:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228248.881119:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec5c4aea x1846777720738048/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228248.881148:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228248.881151:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228248.881177:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720737856:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228248.881266:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720737984:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228248.881297:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761228248.881301:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720737920:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228248.881405:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720738048:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:2.0:1761228248.883645:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228248.883656:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228248.883664:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720738048:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228248.884555:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720737984:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228248.884582:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720737920:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228248.888989:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720737856:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228248.944884:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228249.456228:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228249.967944:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228250.481341:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228250.994714:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228251.507365:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228252.016132:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228252.528196:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228253.040023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228253.552520:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228254.000119:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228254.000128:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228254.000132:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228254.000146:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3847 00000100:00080000:2.0:1761228254.000150:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228254.000159:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228254.000172:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228254.000191:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a37cdda5 x1846777720738304/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228254.000234:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228254.000238:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228254.000452:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720738304:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228254.000547:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720738176:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228254.000593:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761228254.000599:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720738112:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228254.000623:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720738240:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228254.012802:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720738240:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228254.012828:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720738176:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228254.013116:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228254.013121:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228254.013125:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720738304:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228254.013186:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720738112:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228254.064748:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228254.576112:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228255.091487:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228255.600465:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228256.113403:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228256.635101:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228257.136008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228257.649250:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228258.160020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228258.675831:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228259.120085:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228259.120094:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228259.120099:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228259.120111:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3852 00000100:00080000:2.0:1761228259.120115:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228259.120123:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228259.120135:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228259.120149:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084e78951 x1846777720738560/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228259.120171:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228259.120174:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228259.120260:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720738368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228259.120368:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720738496:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228259.120400:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228259.120405:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720738432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228259.120559:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720738560:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228259.130605:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720738368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228259.131086:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720738432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228259.131112:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720738496:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:1.0:1761228259.131568:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:1.0:1761228259.140220:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228259.140231:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228259.140242:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720738560:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228259.185474:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228259.697390:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228260.208047:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228260.722393:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228261.232007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228261.747120:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228262.256034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228262.768171:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228263.281156:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228263.793099:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228264.240160:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228264.240168:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228264.240172:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228264.240183:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3857 00000100:00080000:1.0:1761228264.240186:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761228264.240193:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228264.240203:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228264.240234:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abc685ce x1846777720738816/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228264.240260:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228264.240263:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228264.240300:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720738688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228264.240386:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1761228264.240392:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720738624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228264.240415:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720738752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228264.240495:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720738816:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228264.242503:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720738752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228264.242522:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720738688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228264.244004:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720738624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228264.248836:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228264.248848:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228264.248893:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720738816:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228264.303988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228264.816276:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228265.327997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228265.840159:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228266.354117:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228266.864044:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228267.376023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228267.890234:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228268.400034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228268.913628:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228269.361886:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228269.361895:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228269.361899:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228269.361911:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3862 00000100:00080000:1.0:1761228269.361915:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761228269.361921:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228269.361931:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228269.361948:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720739072/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228269.361976:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228269.362000:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228269.362096:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720739072:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228269.363447:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720738944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228269.363691:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761228269.363699:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720738880:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228269.363726:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720739008:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228269.366578:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228269.366590:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228269.366599:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720739072:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228269.367910:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720738880:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228269.368899:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720739008:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228269.368912:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720738944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228269.423987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228269.936486:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228270.450357:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228270.960721:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228271.472029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228271.989098:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228272.496725:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228273.008067:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228273.521833:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228274.032195:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228274.480103:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228274.480114:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228274.480118:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228274.480132:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3867 00000100:00100000:3.0:1761228274.480133:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720739136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228274.480137:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228274.480147:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228274.480161:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228274.480181:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000790fc921 x1846777720739328/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228274.480205:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228274.480209:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228274.480235:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720739200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228274.480712:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720739328:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228274.482066:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720739264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228274.498267:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720739136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228274.498295:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720739264:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228274.498659:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720739200:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228274.499732:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228274.499742:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228274.499760:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720739328:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228274.547194:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228275.056409:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228275.568034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228276.080130:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228276.592913:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228277.104182:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228277.616477:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228278.129503:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228278.640905:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228279.154232:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228279.603375:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720739392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228279.607233:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720739392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228279.609362:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228279.609369:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228279.609374:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228279.609384:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3872 00000100:00080000:0.0:1761228279.609388:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00100000:1.0:1761228279.609397:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720739456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228279.609398:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228279.609411:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228279.609430:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d88fcfe6 x1846777720739584/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228279.609458:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228279.609462:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228279.609489:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720739520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228279.609780:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720739584:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228279.613936:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720739520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228279.614562:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720739456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228279.616813:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228279.616825:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228279.616836:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720739584:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228279.664271:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228280.176262:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228280.688007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228281.200541:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228281.712092:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228282.224566:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228282.737058:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228283.248276:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228283.760713:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228284.272911:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228284.720085:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228284.720093:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228284.720097:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228284.720108:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3877 00000100:00080000:0.0:1761228284.720112:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228284.720118:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228284.720129:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228284.720144:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a99a8bd5 x1846777720739840/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228284.720167:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228284.720171:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228284.720259:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720739648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228284.720364:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720739776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228284.720403:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228284.720408:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720739712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228284.721797:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720739840:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228284.728756:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720739712:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228284.728775:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720739648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228284.728808:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720739776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228284.729054:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228284.729060:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228284.729064:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720739840:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228284.784797:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228285.299522:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228285.808015:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228286.329514:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228286.832050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228287.344194:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228287.862344:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228288.368707:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228288.880030:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228289.392072:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228289.840369:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228289.840377:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228289.840381:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228289.840391:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3882 00000100:00080000:3.0:1761228289.840394:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228289.840400:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228289.840411:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228289.840414:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720739904:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228289.840427:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8803538 x1846777720740096/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228289.840466:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228289.840470:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228289.840491:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720739968:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228289.840517:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720740032:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228289.842632:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720740096:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228289.846630:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720739904:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228289.846630:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720739968:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228289.846648:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720740032:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228289.848107:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228289.848119:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228289.848129:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720740096:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228289.905195:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228290.418121:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228290.928124:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228291.440360:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228291.952192:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228292.464503:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228292.976552:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228293.489417:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228294.000014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228294.514223:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228294.960175:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228294.960184:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228294.960188:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228294.960199:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3888 00000100:00080000:0.0:1761228294.960203:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228294.960209:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228294.960220:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228294.960237:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720740352/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228294.960264:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228294.960267:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228294.960365:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720740352:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228294.960793:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720740160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228294.960842:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720740288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228294.960901:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228294.960907:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720740224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228294.965007:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720740224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228294.965028:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720740288:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228294.968277:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228294.968290:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228294.968300:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720740352:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228294.968909:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720740160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228295.027157:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228295.538096:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228296.048712:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228296.560186:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228297.072007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228297.584132:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228298.095996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228298.610092:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228299.129777:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228299.636390:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228300.080169:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720740416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228300.081388:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720740480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228300.082434:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761228300.082454:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720740544:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228300.082766:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228300.082772:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228300.082776:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228300.082797:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3893 00000100:00080000:3.0:1761228300.082800:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228300.082805:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228300.082817:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228300.082833:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092860b64 x1846777720740608/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228300.082898:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720740608:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761228300.084138:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228300.084146:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228300.088282:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720740416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228300.089984:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720740480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228300.092693:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720740544:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228300.092775:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228300.092779:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228300.092784:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720740608:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228300.145667:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228300.658650:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228301.168161:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228301.680666:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228302.191985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228302.704054:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228303.216045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228303.730138:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228304.239973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228304.752039:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228305.200558:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228305.200571:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228305.200576:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228305.200588:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3898 00000100:00080000:2.0:1761228305.200593:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:3.0:1761228305.200600:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720740672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228305.200602:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228305.200616:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228305.200633:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fab9894 x1846777720740864/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228305.200660:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228305.200663:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228305.200692:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720740736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228305.200710:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720740800:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228305.201083:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720740864:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228305.204475:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720740672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228305.206519:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720740736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228305.207563:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720740800:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228305.212898:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228305.212929:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228305.213783:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720740864:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228305.264544:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228305.777518:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228306.287999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228306.806659:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228307.312100:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228307.824178:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228308.335988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228308.848530:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228309.363514:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228309.872175:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228310.322209:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228310.322218:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228310.322222:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228310.322235:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3903 00000100:00080000:0.0:1761228310.322238:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228310.322248:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228310.322258:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228310.322278:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000085a527fc x1846777720741120/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228310.322303:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228310.322306:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228310.322417:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720740928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228310.322518:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720741056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228310.322552:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228310.322558:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720740992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228310.322755:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720741120:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228310.328630:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720740992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228310.328653:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720740928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228310.328664:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720741056:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228310.329134:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228310.329155:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228310.329160:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720741120:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228310.384254:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228310.898900:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228311.407943:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228311.920050:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228312.435163:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228312.944105:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228313.455997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228313.968165:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228314.479994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228314.992324:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228315.440161:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720741184:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228315.441441:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228315.441448:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228315.441451:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228315.441459:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3908 00000100:00080000:3.0:1761228315.441463:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228315.441469:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228315.441478:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228315.441494:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e86c3a42 x1846777720741376/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228315.441566:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720741376:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228315.442724:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720741248:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228315.445024:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228315.445036:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228315.445097:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720741312:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228315.457762:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720741184:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228315.462509:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228315.462521:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228315.462532:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720741376:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228315.466255:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720741248:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228315.466438:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720741312:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228315.503994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228316.016729:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228316.528039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228317.040050:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228317.551986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228318.064845:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228318.575994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228319.091244:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228319.600018:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228320.113718:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228320.561286:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228320.561294:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228320.561298:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228320.561308:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3913 00000100:00080000:2.0:1761228320.561312:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228320.561319:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228320.561329:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228320.561345:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006bca6691 x1846777720741632/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228320.561365:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228320.561369:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228320.561395:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720741440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228320.561687:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720741504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228320.561994:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720741632:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228320.562536:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720741568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228320.570841:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720741504:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228320.571643:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228320.571651:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228320.571658:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720741632:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228320.572560:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720741440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228320.574033:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720741568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228320.624450:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228321.136035:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228321.647989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228322.163611:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228322.672954:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228323.184049:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228323.696040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228324.208093:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228324.719993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228325.234974:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228325.680084:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228325.680093:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228325.680097:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228325.680107:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3918 00000100:00080000:0.0:1761228325.680110:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228325.680116:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228325.680127:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228325.680152:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000de75897f x1846777720741888/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228325.680216:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720741888:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228325.681247:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720741696:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228325.681299:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720741824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228325.681328:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228325.681335:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720741760:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228325.682696:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228325.682703:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228325.684915:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720741760:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228325.685549:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720741696:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228325.685568:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720741824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228325.685716:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228325.685734:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228325.685738:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720741888:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228325.745307:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228326.256796:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228326.767987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228327.280096:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228327.791996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228328.305371:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228328.820085:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228329.328058:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228329.840090:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228330.352368:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228330.800067:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228330.800100:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00100000:1.0:1761228330.800103:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720741952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228330.800105:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228330.800113:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3923 00000100:00080000:0.0:1761228330.800120:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228330.800126:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228330.800137:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228330.800157:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720742144/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228330.800178:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228330.800181:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228330.800204:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720742080:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228330.800244:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228330.800253:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720742016:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228330.800373:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720742144:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228330.803494:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720742080:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228330.803696:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720742016:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228330.803708:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720741952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228330.803784:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228330.803790:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228330.803795:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720742144:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228330.864192:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228331.376162:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228331.887990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228332.401170:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228332.911983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228333.425638:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228333.936240:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228334.449182:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228334.960072:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228335.472874:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228335.920753:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720742208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228335.922279:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228335.926294:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228335.926310:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228335.928500:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228335.928511:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228335.928515:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228335.928524:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3929 00000100:00080000:2.0:1761228335.928527:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228335.928533:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228335.928546:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228335.928575:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000003513afe x1846777720742400/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228335.928629:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720742400:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761228335.929682:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228335.929689:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228335.930554:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720742208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228335.939682:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742336:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228335.939703:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228335.944496:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228335.944507:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228335.944517:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720742400:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228335.984371:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228336.499886:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228337.009596:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228337.520157:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228338.032105:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228338.544566:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228339.057892:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228339.568969:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228340.079949:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228340.593052:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228341.041253:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228341.041262:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228341.041265:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228341.041276:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3934 00000100:00080000:3.0:1761228341.041279:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228341.041284:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228341.041294:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228341.041308:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f64df33e x1846777720742656/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228341.041378:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720742656:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228341.041510:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720742464:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228341.041562:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228341.041605:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761228341.041610:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228341.043282:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228341.043290:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228341.046895:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228341.046914:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720742464:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228341.046924:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228341.048389:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228341.048399:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228341.048407:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720742656:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228341.103981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228341.616473:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228342.128007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228342.640149:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228343.153276:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228343.664982:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228344.178714:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228344.688113:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228345.200017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228345.712582:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228346.160886:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228346.160897:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228346.160910:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228346.160921:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3939 00000100:00080000:2.0:1761228346.160924:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228346.160930:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228346.160941:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228346.160957:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003ba27032 x1846777720742912/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228346.160981:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228346.160984:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228346.161011:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228346.161022:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720742912:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228346.161113:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720742720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228346.162376:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720742848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228346.164559:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228346.164571:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228346.164580:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720742912:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228346.175761:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720742784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228346.176009:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720742720:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228346.176021:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720742848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228346.223986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228346.739583:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228347.248022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228347.760468:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228348.275400:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228348.784158:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228349.296004:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228349.809947:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228350.319977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228350.832274:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228351.280208:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228351.280215:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228351.280220:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228351.280232:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3944 00000100:00080000:0.0:1761228351.280236:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228351.280244:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228351.280256:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228351.280274:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720743168/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228351.280294:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228351.280297:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228351.280353:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720743168:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228351.280716:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720742976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228351.280768:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228351.280798:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228351.280803:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228351.283755:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228351.283785:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228351.283797:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720743168:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228351.286936:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720742976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228351.290799:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228351.290820:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743104:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228351.345348:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228351.856659:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228352.367999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228352.882115:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228353.392011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228353.904114:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228354.415987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228354.929617:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228355.439976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228355.952316:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228356.400131:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720743232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228356.400471:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228356.400541:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228356.400562:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228356.400566:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228356.400571:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3949 00000100:00080000:0.0:1761228356.400574:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228356.400579:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228356.400591:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228356.400619:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720743424/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228356.400641:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228356.400644:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228356.400660:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228356.400667:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228356.404660:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720743424:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228356.405371:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720743232:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228356.405415:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228356.408775:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228356.408795:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228356.408805:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720743424:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228356.414293:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228356.463995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228356.976911:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228357.488008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228358.000591:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228358.512026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228359.025273:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228359.536053:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228360.048479:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228360.560785:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228361.072136:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228361.520272:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228361.520280:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228361.520284:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228361.520293:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3954 00000100:00100000:1.0:1761228361.520293:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720743488:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228361.520297:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228361.520305:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228361.520372:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228361.520381:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743616:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228361.520392:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000525c9db8 x1846777720743680/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228361.520416:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:0.0:1761228361.520419:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228361.520423:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228361.520424:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743552:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228361.521119:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720743680:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228361.526124:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720743488:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228361.530199:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743552:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228361.530220:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720743616:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228361.530746:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228361.530752:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228361.530757:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720743680:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228361.586056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228362.096428:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228362.608259:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228363.121905:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228363.634238:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228364.144228:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228364.656021:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228365.173045:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228365.679980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228366.194036:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228366.642283:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228366.642290:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228366.642295:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228366.642308:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3959 00000100:00080000:3.0:1761228366.642312:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228366.642320:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228366.642332:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228366.642348:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008dec2428 x1846777720743936/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228366.642451:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720743936:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228366.642771:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720743744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228366.642897:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720743872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228366.642944:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761228366.642953:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720743808:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228366.643073:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228366.643079:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228366.652537:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720743808:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228366.652559:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720743872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228366.653464:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228366.653473:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228366.653480:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720743936:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228366.653681:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720743744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228366.704677:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228367.217073:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228367.730525:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228368.240606:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228368.751967:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228369.265134:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228369.777122:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228370.288487:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228370.799991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228371.312218:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228371.760691:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228371.760724:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720744000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228371.760734:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228371.760741:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720744064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228371.760848:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228371.760874:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228371.760883:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228371.760889:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3964 00000100:00080000:1.0:1761228371.760893:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228371.760900:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:1.0:1761228371.760909:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228371.760916:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720744128:192.168.202.114@tcp:400:kworker/u8:3.0 00000008:00100000:1.0:1761228371.760959:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228371.760980:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a2fc13a x1846777720744192/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228371.761004:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228371.761007:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228371.761406:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720744192:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228371.763301:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720744064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228371.763596:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720744000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228371.764140:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228371.764156:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228371.764163:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720744192:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228371.766531:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720744128:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228371.824025:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228372.337616:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228372.848884:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228373.361248:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228373.872007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228374.384226:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228374.896224:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228375.413759:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228375.920827:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228376.433255:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228376.881699:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228376.881708:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228376.881712:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228376.881723:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3970 00000100:00080000:3.0:1761228376.881726:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228376.881733:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228376.881746:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228376.881763:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720744448/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228376.881789:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228376.881792:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228376.881834:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720744256:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228376.881944:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720744384:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228376.881978:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228376.881988:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720744320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228376.882376:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720744448:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761228376.888951:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228376.888967:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228376.888980:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720744448:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228376.889411:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720744320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228376.889426:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720744256:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228376.889434:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720744384:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228376.944685:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228377.459368:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228377.968206:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228378.481930:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228378.992094:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228379.504704:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228380.015998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228380.528337:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228381.040019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228381.552112:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228382.002131:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228382.002139:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228382.002143:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228382.002155:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3975 00000100:00080000:0.0:1761228382.002158:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228382.002165:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228382.002177:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228382.002192:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720744704/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228382.002256:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720744704:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761228382.002616:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228382.002620:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228382.002747:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720744512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228382.003319:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720744576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228382.003386:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720744640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228382.012098:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720744576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228382.012209:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720744512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228382.015046:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228382.015056:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228382.015065:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720744704:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228382.015375:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720744640:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228382.064019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228382.576115:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228383.087976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228383.600113:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228384.111996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228384.625785:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228385.135944:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228385.648082:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228386.160014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228386.672119:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228387.120268:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720744768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228387.120959:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228387.120968:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720744832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228387.122115:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228387.122124:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228387.122130:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228387.122139:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3980 00000100:00080000:1.0:1761228387.122144:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228387.122167:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228387.122181:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228387.122201:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abc685ce x1846777720744960/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228387.122294:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720744960:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761228387.122653:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228387.122657:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228387.122683:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228387.122697:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720744896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228387.130387:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720744832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228387.130406:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720744768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228387.132141:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720744896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228387.132602:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228387.132607:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228387.132614:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720744960:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228387.184022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228387.696966:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228388.208019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228388.721600:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228389.232032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228389.744116:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228390.256022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228390.768321:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228391.281113:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228391.792235:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228392.245942:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228392.245951:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228392.245955:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228392.245968:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3985 00000100:00080000:3.0:1761228392.245972:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228392.245981:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228392.245996:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228392.246015:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003d27cd56 x1846777720745216/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228392.246053:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228392.246056:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228392.246184:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720745024:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228392.246286:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720745152:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228392.246323:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228392.246329:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720745088:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228392.246637:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720745216:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761228392.252162:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228392.252176:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228392.252186:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720745216:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228392.253216:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720745024:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228392.253237:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720745152:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228392.253259:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720745088:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228392.304065:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228392.816249:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228393.328041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228393.840639:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228394.352076:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228394.864380:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228395.376031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228395.889341:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228396.400023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228396.912406:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228397.360162:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228397.360172:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228397.360177:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228397.360192:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3990 00000100:00080000:2.0:1761228397.360196:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228397.360204:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228397.360218:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228397.360239:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec5c4aea x1846777720745472/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228397.360264:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228397.360268:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228397.360378:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720745280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228397.360491:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228397.360528:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228397.360536:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228397.360661:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720745472:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228397.365933:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745344:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228397.365980:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720745280:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228397.365992:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228397.366315:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228397.366320:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228397.366325:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720745472:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228397.424139:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228397.939786:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228398.448209:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228398.960090:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228399.472153:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228399.985165:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228400.500236:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228401.008176:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228401.520020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228402.033008:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228402.480465:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228402.480473:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228402.480478:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228402.480490:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 3995 00000100:00080000:3.0:1761228402.480494:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228402.480501:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228402.480515:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228402.480533:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d56d40dd x1846777720745728/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228402.480606:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720745728:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761228402.482154:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228402.482164:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228402.482198:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745600:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228402.482426:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761228402.482434:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720745536:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228402.482471:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745664:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228402.484053:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228402.484064:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228402.484072:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720745728:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228402.487723:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720745536:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228402.487752:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745664:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228402.487766:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745600:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228402.544055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228403.059157:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228403.568022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228404.080777:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228404.592030:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228405.107388:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228405.616033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228406.128070:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228406.640106:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228407.152751:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228407.600168:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720745792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228407.600347:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228407.600355:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228407.600475:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228407.600481:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228407.600485:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228407.600491:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4000 00000100:00080000:2.0:1761228407.600493:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228407.600499:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228407.600510:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228407.600527:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720745984/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228407.600600:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720745984:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228407.600786:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228407.601461:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228407.601468:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228407.604253:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228407.604274:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720745856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228407.604287:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720745792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228407.605018:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228407.605026:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228407.605033:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720745984:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228407.663993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228408.178288:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228408.688367:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228409.202993:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228409.712140:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228410.229268:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228410.738511:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228411.248910:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228411.764530:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228412.278838:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228412.720482:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228412.720491:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228412.720495:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228412.720506:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4005 00000100:00080000:0.0:1761228412.720509:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228412.720515:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228412.720526:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228412.720541:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720746240/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228412.720565:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228412.720568:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228412.720645:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720746240:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228412.721153:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720746048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228412.721207:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228412.721234:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228412.721240:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228412.723523:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746112:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228412.723542:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746176:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228412.724905:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720746048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:2.0:1761228412.724997:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:2.0:1761228412.725005:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228412.725009:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228412.725012:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720746240:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228412.786847:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228413.296167:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228413.808643:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228414.320226:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228414.832011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228415.344423:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228415.857034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228416.368402:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228416.881124:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228417.392240:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228417.840356:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720746304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228417.844686:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228417.845435:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228417.845442:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228417.845446:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228417.845454:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4010 00000100:00080000:0.0:1761228417.845458:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228417.845465:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228417.845478:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228417.845495:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ad2be88 x1846777720746496/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228417.845509:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228417.845529:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228417.845580:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228417.845590:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228417.846501:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720746496:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228417.848166:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228417.848762:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228417.849237:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228417.849243:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228417.849248:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720746496:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228417.849321:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720746304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228417.904028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228418.416110:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228418.927988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228419.440255:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228419.952227:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228420.473207:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228420.976124:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228421.491682:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228421.999962:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228422.515118:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228422.960891:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720746560:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228422.961757:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228422.961763:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228422.961766:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228422.961795:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4016 00000100:00080000:1.0:1761228422.961797:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228422.961803:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228422.961815:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228422.961829:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dc5f36b1 x1846777720746752/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228422.961910:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720746752:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228422.962496:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720746624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228422.962589:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228422.962601:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720746688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228422.963136:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228422.963144:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228422.971262:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720746688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228422.971282:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720746624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228422.971479:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720746560:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228422.971562:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228422.971566:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228422.971571:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720746752:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228423.024027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228423.536130:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228424.048585:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228424.561318:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228425.072016:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228425.586797:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228426.096156:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228426.611311:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228427.120061:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228427.632314:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228428.080936:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228428.080958:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228428.080963:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228428.080975:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4021 00000100:00080000:0.0:1761228428.080979:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228428.080987:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228428.081014:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228428.081036:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d991e6ad x1846777720747008/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228428.081056:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228428.081060:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228428.081083:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720747008:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228428.083039:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228428.083174:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761228428.083183:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720746816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228428.083213:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746944:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228428.090455:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:1.0:1761228428.090483:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720746816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228428.090501:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746944:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228428.090512:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720746880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228428.090777:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228428.090783:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720747008:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228428.144921:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228428.657189:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228429.168014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228429.680080:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228430.193065:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228430.704095:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228431.217417:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228431.728136:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228432.240020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228432.752214:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228433.201797:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720747072:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228433.202279:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228433.202303:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228433.202307:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228433.202314:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4026 00000100:00080000:1.0:1761228433.202318:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228433.202325:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228433.202340:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228433.202362:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720747264/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228433.202450:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720747264:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228433.203370:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720747200:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228433.203405:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228433.203412:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720747136:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228433.204073:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228433.204079:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228433.211683:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720747200:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228433.211726:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720747136:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228433.212056:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228433.212061:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228433.212066:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720747264:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228433.216403:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720747072:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228433.264041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228433.776137:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228434.288145:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228434.800134:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228435.312724:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228435.830440:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228436.337277:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228436.848101:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228437.361265:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228437.872781:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228438.320641:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228438.320654:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228438.320659:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228438.320671:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4031 00000100:00080000:2.0:1761228438.320675:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228438.320682:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:3.0:1761228438.320689:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720747328:192.168.202.114@tcp:400:kworker/u8:2.0 00000008:00100000:2.0:1761228438.320695:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228438.320714:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046c2349c x1846777720747520/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228438.320741:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228438.320745:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228438.320795:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720747392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228438.320802:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720747456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228438.320972:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720747520:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228438.326556:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720747392:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228438.326681:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228438.326687:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228438.326691:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720747520:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228438.326913:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720747328:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228438.327102:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720747456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228438.384902:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228438.896955:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228439.407985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228439.920386:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228440.432050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228440.944911:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228441.456215:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228441.969064:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228442.483007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228442.992247:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228443.442966:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228443.442975:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228443.442979:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228443.442991:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4036 00000100:00080000:3.0:1761228443.442994:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228443.443001:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:2.0:1761228443.443046:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720747584:192.168.202.114@tcp:400:kworker/u8:3.0 00000008:00100000:3.0:1761228443.443130:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228443.443153:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000006d85789 x1846777720747776/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228443.443256:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720747776:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228443.444537:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720747712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228443.444647:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228443.444657:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720747648:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228443.444686:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228443.444694:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228443.455435:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720747648:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228443.455455:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720747584:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228443.455465:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720747712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228443.458446:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228443.458482:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228443.458495:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720747776:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228443.509157:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228444.019453:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228444.527979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228445.040248:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228445.552006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228446.070999:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228446.575995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228447.088276:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228447.599973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228448.112147:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228448.560115:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228448.560124:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228448.560148:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228448.560160:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4041 00000100:00080000:0.0:1761228448.560164:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228448.560171:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228448.560184:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228448.560201:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720748032/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228448.560232:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228448.560235:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228448.560320:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720748032:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228448.560666:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720747840:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228448.560728:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720747968:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228448.560756:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228448.560762:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720747904:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228448.567152:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228448.567165:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228448.567176:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720748032:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228448.567282:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720747904:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228448.567295:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720747968:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228448.568114:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720747840:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228448.624002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228449.136214:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228449.647987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228450.161983:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228450.671994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228451.184178:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228451.696981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228452.210206:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228452.719993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228453.232034:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228453.680262:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228453.680272:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228453.680276:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228453.680289:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4046 00000100:00080000:2.0:1761228453.680311:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228453.680318:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228453.680329:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228453.680344:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006be78503 x1846777720748288/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228453.680371:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228453.680374:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228453.680400:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720748160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228453.680435:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720748288:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228453.680507:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761228453.680515:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720748096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228453.680543:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720748224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228453.685019:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720748160:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228453.686219:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720748224:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228453.687424:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228453.687433:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228453.687439:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720748288:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228453.687602:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720748096:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228453.744033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228454.257536:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228454.767983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228455.280075:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228455.792143:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228456.304243:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228456.816088:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228457.328063:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228457.839996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228458.352433:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228458.803984:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228458.803993:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228458.803997:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228458.804010:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4051 00000100:00080000:3.0:1761228458.804014:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228458.804049:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228458.804063:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228458.804080:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720748544/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228458.804103:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228458.804107:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228458.804137:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720748416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228458.804229:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761228458.804236:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720748352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228458.804261:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720748480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228458.805520:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720748544:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228458.807604:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720748352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228458.807633:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720748480:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228458.807646:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720748416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228458.811196:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228458.811209:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228458.811219:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720748544:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228458.864883:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228459.377620:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228459.890166:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228460.405494:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228460.912007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228461.424825:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228461.937229:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228462.450916:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228462.962511:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228463.472820:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228463.920556:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1761228463.920556:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720748608:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228463.920566:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228463.920572:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228463.920580:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4057 00000100:00080000:0.0:1761228463.920584:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228463.920592:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228463.920607:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228463.920627:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003e02fcda x1846777720748800/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228463.920665:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228463.920670:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228463.920720:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720748672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228463.920901:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228463.920910:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720748736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228463.920973:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720748800:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761228463.928155:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228463.928171:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228463.928185:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720748800:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228463.928252:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720748608:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228463.930951:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720748736:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228463.930987:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720748672:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228463.985989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228464.496653:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228465.008477:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228465.520205:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228466.032044:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228466.544598:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228467.056036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228467.568172:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228468.080198:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228468.594513:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228469.041334:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228469.041341:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228469.041346:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761228469.042278:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720748864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228469.043007:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4062 00000100:00080000:2.0:1761228469.043014:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228469.043021:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228469.043037:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228469.043071:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006be78503 x1846777720749056/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228469.043159:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720749056:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761228469.043790:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228469.043797:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228469.043834:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720748928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228469.044404:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720748992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228469.053697:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720748864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228469.053728:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720748992:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228469.054497:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228469.054505:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228469.054512:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720749056:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228469.056950:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720748928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228469.103952:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228469.617271:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228470.128056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228470.640473:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228471.152028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228471.664607:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228472.176076:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228472.688143:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228473.200518:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228473.712149:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228474.161910:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228474.161918:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228474.161923:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228474.161936:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4067 00000100:00080000:0.0:1761228474.161940:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228474.161947:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228474.161959:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228474.161977:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035650445 x1846777720749312/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228474.161989:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228474.161992:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228474.162367:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720749120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228474.162495:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749248:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228474.162535:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228474.162541:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749184:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228474.162633:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720749312:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761228474.177815:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228474.177831:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228474.177843:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720749312:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228474.178631:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749184:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228474.178670:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749248:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228474.178888:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720749120:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228474.224179:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228474.737548:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228475.248014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228475.761673:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228476.272035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228476.784937:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228477.296156:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228477.809124:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228478.321812:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228478.832045:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228479.280013:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228479.280023:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228479.280044:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228479.280055:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4072 00000100:00080000:0.0:1761228479.280058:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228479.280065:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228479.280075:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228479.280092:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7988e14 x1846777720749568/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228479.280116:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228479.280120:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228479.280152:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720749568:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228479.280189:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720749376:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228479.280249:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720749504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228479.280274:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228479.280280:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720749440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228479.283613:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228479.283627:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228479.283636:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720749568:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228479.284175:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720749440:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228479.284189:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720749504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228479.286907:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720749376:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228479.345646:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228479.856033:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228480.367993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228480.880115:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228481.391980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228481.905177:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228482.415995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228482.928973:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228483.441749:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228483.952240:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228484.400116:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720749632:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228484.401014:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228484.401020:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228484.401023:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228484.401030:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4077 00000100:00080000:0.0:1761228484.401033:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228484.401038:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228484.401048:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228484.401064:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720749824/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228484.401077:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228484.401080:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228484.403911:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720749824:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228484.404400:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749760:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228484.404436:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228484.404442:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749696:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228484.410171:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720749632:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228484.412930:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749696:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228484.412951:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749760:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228484.413056:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228484.413061:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228484.413066:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720749824:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228484.464266:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228484.976045:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228485.487985:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228486.001939:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228486.512756:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228487.025956:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228487.536936:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228488.051925:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228488.560586:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228489.072108:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228489.521510:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228489.521536:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228489.521541:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228489.521554:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4082 00000100:00080000:0.0:1761228489.521558:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228489.521565:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228489.521578:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228489.521595:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fe9fa97 x1846777720750080/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228489.521619:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228489.521622:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228489.522144:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720750080:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228489.522534:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720749888:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228489.522596:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720750016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228489.522602:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749952:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228489.525603:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720749888:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228489.525630:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720750016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228489.529141:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720749952:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228489.529515:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228489.529632:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228489.529637:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720750080:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228489.584711:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228490.096144:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228490.613113:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228491.120268:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228491.631991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228492.144052:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228492.656806:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228493.168065:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228493.680046:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228494.192754:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228494.640124:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228494.640133:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228494.640138:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228494.640151:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4087 00000100:00100000:1.0:1761228494.640153:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720750144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228494.640166:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228494.640176:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228494.640188:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228494.640207:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000129fdcbe x1846777720750336/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228494.640256:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228494.640261:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228494.640263:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228494.640303:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228494.640309:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228494.641024:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720750336:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228494.645528:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228494.645547:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720750144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228494.645557:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750272:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228494.647845:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228494.647883:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228494.647894:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720750336:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228494.704943:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228495.217190:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228495.727982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228496.240940:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228496.753625:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228497.264278:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228497.776331:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228498.291613:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228498.799984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228499.312081:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228499.760127:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228499.760157:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228499.760161:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228499.760174:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4092 00000100:00080000:1.0:1761228499.760179:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228499.760188:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228499.760202:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228499.760230:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086fe0f7b x1846777720750592/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228499.760260:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228499.760264:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228499.760333:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720750592:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228499.760627:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750464:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228499.760681:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761228499.760688:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720750400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228499.760712:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750528:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228499.769520:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228499.769536:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228499.769549:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720750592:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228499.769635:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750464:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228499.769655:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720750400:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228499.769666:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750528:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228499.825345:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228500.338164:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228500.848001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228501.367197:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228501.872533:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228502.389662:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228502.896387:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228503.408712:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228503.920034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228504.432145:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228504.880808:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720750656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228504.881106:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228504.881272:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228504.881277:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228504.881288:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4098 00000100:00080000:1.0:1761228504.881292:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228504.881300:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228504.881312:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228504.881332:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057d87282 x1846777720750848/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228504.881347:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228504.881371:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228504.881595:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720750720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228504.881681:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228504.881687:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720750784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228504.881942:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720750848:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228504.883564:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720750656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228504.891254:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228504.891270:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228504.891283:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720750848:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228504.891428:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720750784:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228504.891442:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720750720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228504.944050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228505.456626:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228505.967927:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228506.482006:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228506.992265:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228507.504146:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228508.016040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228508.529666:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228509.040997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228509.552229:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228510.000112:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228510.000121:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228510.000125:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228510.000139:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4103 00000100:00080000:1.0:1761228510.000143:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761228510.000146:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720750912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228510.000153:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228510.000166:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228510.000186:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005d7eada1 x1846777720751104/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228510.000213:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228510.000217:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228510.000243:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228510.000269:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720751040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228510.000927:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720751104:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761228510.004420:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228510.004432:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228510.004456:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720751104:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228510.004561:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720750976:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228510.004591:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720750912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228510.004601:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720751040:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228510.064020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228510.578457:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228511.088276:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228511.600900:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228512.112893:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228512.624592:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228513.138488:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228513.648663:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228514.159992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228514.672215:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228515.120161:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228515.120169:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228515.120189:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228515.120200:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4108 00000100:00080000:3.0:1761228515.120204:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228515.120210:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228515.120221:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228515.120241:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720751360/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228515.120270:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228515.120274:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228515.120305:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720751232:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228515.120513:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720751168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228515.120580:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720751360:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228515.121668:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720751296:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228515.125765:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720751232:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228515.125955:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228515.125961:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228515.125965:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720751360:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228515.126814:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720751168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228515.126831:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720751296:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228515.184602:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228515.696834:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228516.207993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228516.720092:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228517.232811:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228517.744492:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228518.256182:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228518.769186:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228519.279944:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228519.797959:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228520.241265:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228520.241272:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228520.241276:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228520.241285:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4113 00000100:00080000:1.0:1761228520.241288:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228520.241294:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228520.241303:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228520.241315:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720751616/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228520.241357:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228520.241360:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228520.241383:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720751488:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228520.241469:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761228520.241474:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720751424:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228520.241490:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720751552:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228520.241673:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720751616:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228520.243931:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720751424:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228520.243954:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720751552:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228520.243963:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720751488:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228520.245325:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228520.245334:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228520.245344:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720751616:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228520.303974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228520.817740:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228521.327992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228521.840728:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228522.354129:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228522.864654:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228523.375990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228523.889951:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228524.399992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228524.912180:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228525.361218:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720751680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228525.362116:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228525.362123:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228525.362127:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228525.362134:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4118 00000100:00080000:1.0:1761228525.362137:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228525.362142:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228525.362152:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228525.362166:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000019fe5bef x1846777720751872/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228525.362196:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228525.362199:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228525.362218:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720751744:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228525.362278:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228525.362285:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720751808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228525.362288:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720751872:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228525.365297:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720751680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228525.365729:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720751808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228525.365744:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720751744:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228525.370751:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228525.370774:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228525.372614:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720751872:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228525.425160:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228525.936644:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228526.447976:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228526.961162:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228527.472691:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228527.984122:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228528.498251:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228529.008060:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228529.520194:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228530.033337:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228530.480670:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228530.480679:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228530.480683:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228530.480696:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4123 00000100:00080000:3.0:1761228530.480699:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1761228530.480704:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720751936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228530.480707:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228530.480721:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228530.480750:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000074fca4bf x1846777720752128/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228530.480769:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228530.480773:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228530.480811:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720752064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228530.480820:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720752000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228530.480993:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720752128:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228530.484569:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720752000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228530.484663:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228530.484669:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228530.484673:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720752128:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228530.486286:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720752064:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228530.486314:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720751936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228530.543981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228531.060367:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228531.567973:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228532.084274:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228532.592879:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228533.106504:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228533.615978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228534.129931:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228534.642662:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228535.155025:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228535.603572:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228535.603580:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228535.603584:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228535.603597:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4128 00000100:00080000:0.0:1761228535.603601:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228535.603609:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228535.603623:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228535.603643:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000525c9db8 x1846777720752384/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228535.603658:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228535.603663:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228535.603809:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720752192:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228535.603950:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720752320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228535.603982:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228535.603986:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720752256:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228535.604720:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720752384:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228535.616195:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720752192:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228535.616226:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720752320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228535.616263:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720752256:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228535.621359:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228535.621382:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228535.621394:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720752384:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228535.664512:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228536.183609:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228536.688044:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228537.204616:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228537.712010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228538.224906:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228538.737804:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228539.248118:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228539.762064:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228540.272492:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228540.720200:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228540.720209:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228540.720214:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228540.720226:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4133 00000100:00080000:0.0:1761228540.720229:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228540.720237:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228540.720249:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228540.720269:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018849446 x1846777720752640/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228540.720296:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228540.720300:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228540.720377:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720752448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228540.720501:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720752576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228540.720540:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228540.720544:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720752512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228540.721684:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720752640:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228540.725381:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720752576:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228540.725422:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720752512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228540.725431:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720752448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228540.730792:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228540.730807:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228540.730820:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720752640:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228540.784022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228541.297267:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228541.808584:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228542.320756:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228542.831984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228543.344466:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228543.856101:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228544.368018:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228544.879945:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228545.393018:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228545.840360:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720752704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228545.840583:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228545.840589:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720752768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228545.841455:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228545.841461:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228545.841465:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228545.841471:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4138 00000100:00080000:0.0:1761228545.841490:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228545.841496:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228545.841505:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228545.841520:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fe9fa97 x1846777720752896/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228545.841542:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228545.841545:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228545.841586:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228545.841595:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720752832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228545.841802:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720752896:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228545.850301:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720752704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228545.852338:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228545.852348:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228545.852357:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720752896:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228545.852523:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720752832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228545.852564:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720752768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228545.909071:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228546.417342:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228546.928226:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228547.440050:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228547.953023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228548.464547:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228548.975993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228549.488154:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228550.001981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228550.514342:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228550.961399:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228550.961408:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228550.961411:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228550.961422:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4144 00000100:00080000:2.0:1761228550.961426:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228550.961432:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228550.961443:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228550.961458:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000290da3b x1846777720753152/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228550.961485:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228550.961488:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228550.961551:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720753024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228550.961644:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761228550.961650:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720752960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228550.961674:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720753088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228550.961830:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720753152:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228550.971348:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720752960:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228550.971373:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720753088:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228550.971385:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720753024:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228550.972911:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228550.972923:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228550.972932:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720753152:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228551.023982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228551.536103:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228552.049008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228552.560076:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228553.071924:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228553.584992:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228554.097065:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228554.608567:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228555.120133:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228555.639653:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228556.080055:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228556.080064:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228556.080067:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228556.080077:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4149 00000100:00080000:2.0:1761228556.080080:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228556.080086:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228556.080096:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228556.080111:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046c2349c x1846777720753408/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228556.080137:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228556.080141:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228556.080224:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720753408:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228556.080528:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720753216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228556.080585:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720753344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228556.080632:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228556.080638:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720753280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228556.085828:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720753280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228556.087353:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720753216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228556.087372:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720753344:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228556.087480:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228556.087485:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228556.087490:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720753408:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228556.144001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228556.657692:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228557.167974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228557.680104:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228558.193107:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228558.704661:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228559.215972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228559.728762:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228560.240001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228560.752399:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228561.204099:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228561.204109:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228561.204114:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228561.204125:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4154 00000100:00080000:0.0:1761228561.204128:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228561.204134:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228561.204145:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228561.204179:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b0db11ae x1846777720753664/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228561.204206:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228561.204210:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228561.204239:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720753536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228561.204338:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761228561.204345:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720753472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228561.204371:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720753600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228561.204637:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720753664:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228561.206417:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720753600:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228561.206436:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720753536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228561.207807:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228561.207824:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228561.207831:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720753664:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228561.208504:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720753472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228561.264010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228561.778124:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228562.287981:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228562.800178:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228563.312983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228563.824150:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228564.335951:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228564.848086:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228565.359986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228565.873600:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228566.320938:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228566.320965:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228566.320970:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228566.320980:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4159 00000100:00080000:2.0:1761228566.320984:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228566.320990:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228566.321001:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228566.321017:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720753920/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228566.321033:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228566.321036:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228566.321117:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720753920:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228566.322043:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720753728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228566.322104:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720753856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228566.322136:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228566.322143:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720753792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228566.326502:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720753792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228566.326523:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720753728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228566.329657:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720753856:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:0.0:1761228566.332990:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:0.0:1761228566.333005:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228566.333010:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228566.333014:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720753920:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228566.384153:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228566.896770:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228567.408054:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228567.920067:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228568.432212:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228568.944759:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228569.455995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228569.968057:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228570.480145:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228570.992090:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228571.440762:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720753984:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228571.443544:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228571.443572:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228571.443576:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228571.443586:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4164 00000100:00080000:0.0:1761228571.443589:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228571.443595:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228571.443606:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228571.443624:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002168af17 x1846777720754176/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228571.443640:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228571.443643:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228571.443672:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720754048:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228571.443748:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228571.443754:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720754112:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228571.443782:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720754176:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228571.449308:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720753984:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228571.452077:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228571.452087:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228571.452097:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720754176:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228571.452169:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720754112:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228571.452181:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720754048:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228571.507486:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228572.016070:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228572.528008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228573.040645:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228573.552896:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228574.066529:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228574.576033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228575.088395:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228575.600027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228576.112681:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228576.560419:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720754240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228576.561256:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228576.561282:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720754304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228576.561605:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228576.561611:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228576.561620:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228576.561626:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4169 00000100:00080000:2.0:1761228576.561629:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228576.561635:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228576.561648:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228576.561669:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000064ce9b52 x1846777720754432/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228576.561716:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720754432:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228576.562571:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720754368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228576.566458:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228576.566473:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761228576.566595:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228576.566602:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228576.566612:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720754432:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228576.567388:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720754368:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228576.567409:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720754304:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228576.572885:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720754240:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228576.624053:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228577.136168:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228577.648837:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228578.160229:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228578.672040:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228579.188505:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228579.696038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228580.208141:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228580.720375:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228581.232422:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228581.681257:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228581.681266:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228581.681271:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228581.681285:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4174 00000100:00080000:0.0:1761228581.681289:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1761228581.681292:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720754496:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228581.681299:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228581.681312:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228581.681330:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720754688/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228581.681352:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228581.681356:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228581.681384:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720754560:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228581.681396:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720754624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228581.682105:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720754688:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228581.691897:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720754496:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228581.693153:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720754560:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228581.693199:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720754624:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228581.693300:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228581.693305:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228581.693309:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720754688:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228581.744027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228582.256253:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228582.768007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228583.280480:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228583.792667:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228584.304136:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228584.816030:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228585.328271:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228585.842774:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228586.352363:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228586.802049:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228586.802059:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228586.802063:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228586.802109:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4179 00000100:00080000:2.0:1761228586.802115:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228586.802124:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228586.802138:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228586.802157:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720754752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228586.802163:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000003513afe x1846777720754944/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228586.802210:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228586.802214:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228586.802269:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720754816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228586.802282:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720754880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228586.802561:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720754944:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228586.806545:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720754752:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228586.806572:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720754880:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228586.807422:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228586.807430:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228586.807436:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720754944:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228586.808793:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720754816:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228586.864090:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228587.377998:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228587.888011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228588.400298:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228588.916117:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228589.425277:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228589.935944:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228590.448251:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228590.960282:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228591.472645:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228591.922233:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720755008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228591.924769:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228591.924778:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228591.924783:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228591.924796:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4185 00000100:00080000:2.0:1761228591.924799:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228591.924808:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228591.924822:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228591.924841:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001379e0d8 x1846777720755200/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228591.924941:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720755200:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228591.925894:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720755072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228591.925963:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228591.925971:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720755136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228591.926286:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228591.926292:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228591.927023:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720755008:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228591.936474:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228591.936493:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228591.936506:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720755200:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228591.938232:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720755136:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228591.942364:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720755072:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228591.984039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228592.498743:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228593.008064:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228593.523894:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228594.033261:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228594.544162:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228595.056046:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228595.570442:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228596.080061:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228596.597496:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228597.043834:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228597.043842:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228597.043847:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228597.043886:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4190 00000100:00080000:3.0:1761228597.043891:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228597.043899:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228597.043915:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228597.043936:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035883416 x1846777720755456/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228597.043949:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228597.043962:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228597.044032:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720755328:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228597.044148:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761228597.044155:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720755264:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228597.044183:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720755392:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228597.044735:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720755456:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228597.048095:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720755264:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228597.048119:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720755328:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228597.057416:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720755392:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228597.059653:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228597.059663:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228597.059672:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720755456:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228597.104001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228597.616115:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228598.128799:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228598.640157:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228599.152412:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228599.665105:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228600.175984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228600.688540:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228601.200008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228601.712163:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228602.160257:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228602.160285:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228602.160289:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228602.160300:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4195 00000100:00080000:2.0:1761228602.160304:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228602.160310:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:3.0:1761228602.160420:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720755520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228602.160524:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720755648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228602.160560:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228602.160567:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720755584:192.168.202.114@tcp:400:kworker/u8:2.0 00000008:00100000:2.0:1761228602.160662:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228602.160681:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fab9894 x1846777720755712/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228602.160757:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720755712:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761228602.161244:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228602.161249:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761228602.166871:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228602.166882:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228602.166900:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fab9894 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720755712:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228602.167001:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c987228a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720755584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228602.167011:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006bca6691 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720755520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228602.167019:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007221726f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720755648:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228602.224042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228602.741377:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228603.248005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228603.762080:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228604.276234:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228604.785890:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228605.297171:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228605.808219:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228606.320140:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228606.832704:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228607.280997:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720755776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228607.281386:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228607.281392:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720755840:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228607.281583:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228607.281588:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228607.281591:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228607.281596:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4200 00000100:00080000:0.0:1761228607.281599:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228607.281605:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228607.281614:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228607.281628:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d991e6ad x1846777720755968/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228607.281681:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720755968:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228607.282136:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720755904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228607.282369:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228607.282374:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761228607.289983:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228607.289996:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228607.290007:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720755968:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228607.290115:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720755904:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228607.290148:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720755840:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228607.290157:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720755776:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228607.343978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228607.857899:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228608.368033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228608.881443:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228609.392060:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228609.904903:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228610.415996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228610.931423:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228611.440529:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228611.952058:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228612.400590:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228612.400600:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228612.400604:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228612.400614:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4205 00000100:00080000:1.0:1761228612.400617:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228612.400624:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228612.400634:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228612.400649:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017ee9697 x1846777720756224/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228612.400729:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720756224:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228612.401615:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720756032:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228612.401668:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720756160:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228612.401694:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228612.401700:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720756096:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228612.402093:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228612.402097:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761228612.405531:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228612.405544:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228612.405556:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720756224:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228612.408114:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720756096:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228612.408135:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720756032:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228612.408146:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720756160:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228612.464369:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228612.976061:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228613.487989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228614.000028:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228614.513084:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228615.024268:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228615.536002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228616.048104:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228616.560027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228617.072105:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228617.520092:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228617.520101:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228617.520106:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228617.520121:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4210 00000100:00080000:1.0:1761228617.520125:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228617.520152:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228617.520165:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228617.520174:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720756288:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228617.520193:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017ee9697 x1846777720756480/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228617.520222:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228617.520226:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228617.520254:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720756352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228617.520409:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720756480:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228617.522001:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720756416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228617.528694:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aa6e6fe7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720756352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228617.528755:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012902ca7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720756416:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228617.531316:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228617.531330:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228617.531358:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017ee9697 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720756480:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228617.532762:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000961c7ab0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720756288:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228617.585657:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228618.096781:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228618.607990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228619.120030:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228619.633157:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228620.145913:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228620.656031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228621.169376:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228621.680807:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228622.195193:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228622.640680:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228622.640688:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228622.640713:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228622.640729:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4215 00000100:00080000:0.0:1761228622.640734:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228622.640742:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228622.640758:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228622.640782:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000085a527fc x1846777720756736/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228622.640812:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228622.640816:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228622.640967:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720756544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228622.640984:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720756736:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228622.641051:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720756608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228622.641179:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720756672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228622.651914:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720756544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228622.651998:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720756608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228622.652147:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228622.652153:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228622.652159:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720756736:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228622.653733:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720756672:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228622.703925:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228623.221956:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228623.728011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228624.240242:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228624.752041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228625.265100:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228625.776029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228626.288361:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228626.800151:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228627.312108:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228627.760405:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228627.760413:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228627.760417:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228627.760428:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4220 00000100:00080000:2.0:1761228627.760432:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228627.760438:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228627.760450:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228627.760466:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003ba27032 x1846777720756992/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228627.760491:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228627.760495:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228627.760518:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720756864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228627.760621:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761228627.760627:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720756800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228627.760652:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720756928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228627.761306:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720756992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761228627.769584:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228627.769603:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228627.769612:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ba27032 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720756992:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228627.769672:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720756928:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228627.769700:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720756864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228627.770671:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720756800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228627.825734:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228628.336296:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228628.848154:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228629.360197:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228629.871995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228630.384148:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228630.898421:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228631.409015:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228631.920048:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228632.432498:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228632.880125:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761228632.880135:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720757056:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228632.880135:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228632.880140:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228632.880148:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4226 00000100:00080000:3.0:1761228632.880152:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228632.880159:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228632.880172:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228632.880189:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000341ad5cd x1846777720757248/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228632.880261:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720757248:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228632.880275:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720757184:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228632.880312:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761228632.880319:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720757120:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228632.881080:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228632.881089:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228632.898713:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720757120:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228632.898745:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720757184:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228632.898759:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1761228632.898763:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720757056:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228632.898766:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228632.898792:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720757248:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228632.944030:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228633.456091:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228633.968312:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228634.480842:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228634.992088:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228635.504654:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228636.016041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228636.528115:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228637.040045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228637.552265:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228638.000387:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720757312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228638.002675:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228638.002702:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720757376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228638.002717:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228638.002723:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228638.002728:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228638.002735:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4231 00000100:00080000:0.0:1761228638.002738:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228638.002746:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228638.002759:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228638.002781:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c7988e14 x1846777720757504/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228638.002788:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720757440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228638.002800:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228638.002805:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228638.002901:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720757504:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228638.007447:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720757312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228638.007824:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228638.007831:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228638.008114:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720757440:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228638.008130:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720757376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228638.008220:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720757504:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228638.064113:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228638.576104:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228639.090213:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228639.600710:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228640.112307:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228640.627943:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228641.136429:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228641.649249:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228642.160048:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228642.672214:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228643.120791:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228643.120800:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228643.120804:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228643.120815:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4236 00000100:00080000:0.0:1761228643.120827:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228643.120833:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228643.120845:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228643.120883:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a99a8bd5 x1846777720757760/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228643.120904:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228643.120908:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228643.120940:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720757568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228643.120951:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720757760:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228643.121029:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720757696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228643.121076:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228643.121084:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720757632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228643.128251:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228643.128269:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228643.128522:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720757632:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228643.128537:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720757568:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228643.128548:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720757696:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228643.128726:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720757760:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228643.185166:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228643.696517:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228644.210554:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228644.720055:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228645.232128:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228645.743977:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228646.256159:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228646.768077:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228647.279974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228647.792111:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228648.240422:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720757824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228648.241955:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720757888:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228648.243252:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720757952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228648.244035:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228648.244043:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228648.244046:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228648.244063:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4241 00000100:00080000:1.0:1761228648.244067:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228648.244074:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228648.244085:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228648.244109:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000062f60995 x1846777720758016/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228648.244175:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720758016:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761228648.245270:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228648.245279:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228648.248208:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d0d44bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720757952:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228648.248241:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000019fe5bef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720757824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228648.248298:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720757888:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228648.248594:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228648.248605:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228648.248614:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000062f60995 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720758016:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228648.305027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228648.816752:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228649.328002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228649.840590:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228650.352008:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228650.864500:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228651.376027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228651.888375:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228652.401121:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228652.912410:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228653.360674:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228653.360681:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228653.360685:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228653.360696:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4246 00000100:00080000:3.0:1761228653.360699:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228653.360706:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228653.360715:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228653.360731:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b01aecf x1846777720758272/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228653.360759:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228653.360762:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228653.360844:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720758080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228653.360967:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720758208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228653.361018:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228653.361025:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720758144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228653.361304:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720758272:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228653.370174:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720758144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228653.370192:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720758080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228653.370200:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720758208:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228653.370285:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228653.370300:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228653.370305:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720758272:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228653.425796:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228653.936215:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228654.447995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228654.960269:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228655.471984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228655.988650:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228656.496007:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228657.012807:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228657.519982:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228658.032132:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228658.480093:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228658.480120:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228658.480124:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228658.480137:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4251 00000100:00080000:3.0:1761228658.481228:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228658.481254:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228658.481271:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228658.481303:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f64df33e x1846777720758528/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228658.481329:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228658.481333:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228658.481374:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720758528:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228658.481453:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720758336:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228658.482108:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720758464:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228658.482156:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761228658.482162:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720758400:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228658.485416:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720758464:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228658.485448:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720758400:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228658.486909:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720758336:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228658.486988:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228658.486993:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228658.486997:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720758528:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228658.543990:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228659.057698:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228659.571353:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228660.080128:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228660.592010:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228661.104089:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228661.617541:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228662.128154:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228662.640322:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228663.154936:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228663.601892:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228663.601900:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228663.601905:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228663.601919:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4256 00000100:00080000:3.0:1761228663.601924:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228663.601932:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228663.601944:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228663.601963:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8803538 x1846777720758784/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228663.601985:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228663.602004:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228663.602125:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720758784:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228663.602365:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720758592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228663.602442:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720758720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228663.602470:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228663.602474:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720758656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228663.609206:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228663.609221:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228663.609232:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720758784:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228663.609424:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720758656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228663.609442:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720758592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228663.609452:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720758720:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228663.664249:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228664.176207:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228664.688198:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228665.200060:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228665.713061:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228666.224455:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228666.736027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228667.249063:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228667.760218:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228668.272651:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228668.722714:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228668.722722:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228668.722726:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228668.722737:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4261 00000100:00080000:3.0:1761228668.722742:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228668.722749:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228668.722761:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228668.722778:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c2618360 x1846777720759040/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228668.722790:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228668.722793:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228668.722913:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720759040:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228668.724774:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720758848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228668.724908:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720758976:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228668.724946:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761228668.724954:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720758912:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228668.729321:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228668.729353:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228668.729366:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c2618360 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720759040:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228668.733269:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092860b64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720758912:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228668.733295:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e86c3a42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720758848:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228668.733308:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3a840a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720758976:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228668.784449:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228669.296728:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228669.808014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228670.321121:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228670.832505:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228671.344345:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228671.856031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228672.370344:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228672.880850:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228673.392822:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228673.840619:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228673.840628:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228673.840633:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228673.840645:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4266 00000100:00080000:1.0:1761228673.840649:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228673.840657:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228673.840670:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228673.840690:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001c661ef2 x1846777720759296/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228673.840720:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228673.840724:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228673.840826:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720759104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228673.840981:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720759232:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228673.841018:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228673.841025:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720759168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228673.841041:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720759296:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228673.849913:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007ef613dc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720759104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228673.851552:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d7eada1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720759168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228673.851572:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057d87282 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720759232:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228673.851916:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228673.851923:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228673.851929:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c661ef2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720759296:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228673.904033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228674.416884:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228674.928568:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228675.441387:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228675.951984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228676.464195:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228676.976373:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228677.488832:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228678.000074:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228678.513841:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228678.960441:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:2.0:1761228678.960443:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720759360:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228678.960450:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228678.960456:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228678.960465:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4272 00000100:00080000:3.0:1761228678.960469:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228678.960476:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228678.960490:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228678.960508:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f64df33e x1846777720759552/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228678.960523:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228678.960527:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228678.960578:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720759488:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228678.960611:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228678.960616:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720759424:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228678.960666:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720759552:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228678.970720:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720759360:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228678.977325:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720759488:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228678.977374:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720759424:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228678.977654:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228678.977659:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228678.977664:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720759552:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228679.024363:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228679.538060:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228680.048094:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228680.562504:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228681.072270:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228681.584545:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228682.097459:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228682.608174:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228683.121202:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228683.637468:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228684.080406:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228684.080415:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228684.080419:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228684.080432:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4277 00000100:00080000:3.0:1761228684.080436:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228684.080443:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228684.080472:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228684.080493:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abb9f4e4 x1846777720759808/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228684.080525:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228684.080529:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228684.080589:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720759808:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228684.080711:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720759616:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228684.080760:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720759744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228684.080791:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761228684.080797:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720759680:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228684.089035:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228684.089053:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228684.089066:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720759808:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228684.090946:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720759680:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228684.090968:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720759616:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228684.090980:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720759744:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228684.144037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228684.659044:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228685.168334:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228685.680826:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228686.191952:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228686.705649:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228687.216035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228687.728243:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228688.240674:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228688.752458:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228689.200288:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228689.200296:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228689.200301:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:0.0:1761228689.200313:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720759872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228689.200407:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4282 00000100:00080000:1.0:1761228689.200411:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761228689.200418:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720760000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228689.200421:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228689.200436:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228689.200444:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228689.200450:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720759936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228689.200458:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad2d99f0 x1846777720760064/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761228689.200518:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720760064:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:1.0:1761228689.201002:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228689.201008:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228689.206199:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086fe0f7b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720760000:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228689.206946:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720759936:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228689.206959:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720759872:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228689.207207:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228689.207212:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228689.207217:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad2d99f0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720760064:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228689.264051:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228689.776784:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228690.288052:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228690.800738:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228691.312022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228691.824464:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228692.335988:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228692.848152:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228693.360023:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228693.872622:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228694.321188:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228694.321198:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228694.321202:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228694.321216:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4287 00000100:00080000:2.0:1761228694.321237:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228694.321245:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228694.321269:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228694.321290:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000084e78951 x1846777720760320/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228694.321320:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228694.321324:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228694.321361:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720760192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228694.321374:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720760320:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228694.321689:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720760128:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228694.322256:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720760256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228694.329821:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720760192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228694.329945:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228694.329951:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228694.329955:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084e78951 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720760320:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228694.330170:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001379e0d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720760128:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228694.330186:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003513afe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720760256:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228694.384282:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228694.896204:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228695.408037:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228695.920785:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228696.432143:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228696.944948:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228697.455972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228697.968240:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228698.479987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228698.992982:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228699.440888:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720760384:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228699.443033:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1761228699.443049:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720760448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228699.443765:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228699.443771:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228699.443775:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228699.443781:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4292 00000100:00080000:2.0:1761228699.443785:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228699.443791:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228699.443803:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228699.443819:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c9f23bda x1846777720760576/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228699.443881:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720760576:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228699.444746:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720760512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228699.446880:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228699.446894:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:0.0:1761228699.450996:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228699.451012:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228699.451023:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c9f23bda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720760576:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228699.451199:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec5c4aea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720760448:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228699.453231:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000790fc921 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720760384:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228699.455992:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046bdcfa1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720760512:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228699.504234:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228700.016171:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228700.528024:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228701.040131:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228701.552019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228702.067436:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228702.576526:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228703.088073:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228703.600116:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228704.112186:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228704.560280:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720760640:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228704.560464:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228704.560471:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720760704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228704.561061:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228704.561067:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228704.561073:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228704.561080:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4297 00000100:00080000:2.0:1761228704.561084:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228704.561091:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:3.0:1761228704.561103:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720760768:192.168.202.114@tcp:400:kworker/u8:3.0 00000008:00100000:2.0:1761228704.561105:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228704.561128:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfe71489 x1846777720760832/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228704.561154:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228704.561157:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228704.561199:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720760832:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761228704.564767:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228704.564781:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228704.564793:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfe71489 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720760832:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228704.564803:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000064ce9b52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720760768:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228704.564825:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720760704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228704.564838:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720760640:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228704.625045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228705.136324:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228705.648029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228706.160143:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228706.672033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228707.184511:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228707.698001:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:3.0:1761228708.208640:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228708.720766:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228709.232711:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228709.680251:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228709.680259:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228709.680263:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228709.680275:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4302 00000100:00080000:1.0:1761228709.680279:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1761228709.680282:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720760896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228709.680287:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228709.680297:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228709.680312:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b1564d60 x1846777720761088/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228709.680342:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228709.680346:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228709.680370:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720760960:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228709.680391:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720761088:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228709.680398:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720761024:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228709.682988:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228709.682999:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228709.683007:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1564d60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720761088:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228709.684002:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720760896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228709.684221:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000094e95ca3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720760960:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228709.684291:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720761024:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228709.744110:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228710.256276:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228710.768401:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228711.281922:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228711.792033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228712.304709:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228712.816020:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228713.328185:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228713.839977:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228714.354380:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228714.800075:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228714.800084:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228714.800089:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228714.800103:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4307 00000100:00080000:1.0:1761228714.800122:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228714.800130:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228714.800143:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228714.800163:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a5268d0e x1846777720761344/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228714.800177:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228714.800180:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228714.800220:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720761152:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228714.800330:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720761280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228714.800390:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720761216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228714.802346:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720761344:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228714.806421:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720761216:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228714.806615:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720761152:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228714.806627:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720761280:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228714.806747:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228714.806753:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228714.806758:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720761344:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228714.864179:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228715.376150:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228715.887989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228716.400118:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228716.912017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228717.425812:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228717.936003:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228718.448207:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228718.959989:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228719.473259:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228719.920528:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228719.920535:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228719.920555:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228719.920565:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4313 00000100:00080000:3.0:1761228719.920569:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228719.920575:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228719.920585:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228719.920609:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003d27cd56 x1846777720761600/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228719.920631:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228719.920634:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228719.920658:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720761472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228719.920754:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1761228719.920760:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720761408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228719.920783:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720761536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228719.920844:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720761600:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228719.926818:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720761408:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:02020000:2.0:1761228719.927275:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:2.0:1761228719.927285:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228719.927289:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228719.927293:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720761600:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228719.927370:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720761536:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228719.927383:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720761472:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228719.984045:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228720.498010:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228721.007950:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228721.520796:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228722.031999:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228722.547576:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228723.056002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228723.568572:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228724.080245:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228724.593136:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228725.040541:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228725.040596:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228725.040600:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228725.040610:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4318 00000100:00080000:3.0:1761228725.040614:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1761228725.040620:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228725.040630:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228725.040646:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cd946368 x1846777720761856/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228725.040733:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720761856:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228725.041250:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720761728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228725.041308:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761228725.041315:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720761664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228725.041340:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720761792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228725.044567:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228725.044577:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228725.045206:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720761728:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228725.045230:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720761792:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228725.045724:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228725.045729:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228725.045733:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720761856:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228725.045811:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720761664:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228725.105273:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228725.616145:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228726.136058:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228726.640775:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228727.151971:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228727.665707:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228728.175978:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228728.688160:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228729.199997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228729.712216:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228730.160902:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720761920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228730.161103:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228730.161110:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720761984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228730.161532:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720762048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228730.162300:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228730.162308:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228730.162313:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228730.162320:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4323 00000100:00080000:0.0:1761228730.162324:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1761228730.162332:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228730.162348:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228730.162430:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d88fcfe6 x1846777720762112/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228730.162549:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720762112:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761228730.162898:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228730.162904:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761228730.171396:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228730.171411:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228730.171424:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720762112:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228730.171514:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720762048:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228730.171538:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720761984:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228730.171550:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720761920:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228730.224634:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228730.736724:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228731.248012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228731.760632:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228732.272797:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228732.784074:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228733.296062:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228733.810651:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228734.319984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228734.832069:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228735.281420:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720762176:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228735.283218:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228735.283226:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228735.283231:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228735.283279:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4328 00000100:00080000:2.0:1761228735.283283:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1761228735.283289:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228735.283346:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228735.283364:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046c2349c x1846777720762368/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228735.283431:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720762368:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228735.283447:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720762304:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228735.283695:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228735.283700:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228735.283724:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720762240:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228735.292763:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720762240:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228735.293407:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a37cdda5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720762304:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228735.293420:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720762176:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228735.294160:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228735.294167:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228735.294174:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720762368:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228735.344816:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228735.856606:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228736.368005:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228736.881361:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228737.391979:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228737.904086:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228738.415996:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228738.930178:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228739.440113:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228739.952077:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228740.400548:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720762432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228740.403139:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1761228740.403157:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720762496:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228740.403932:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228740.403938:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228740.403942:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228740.403948:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4333 00000100:00080000:1.0:1761228740.403951:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1761228740.403957:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228740.403968:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228740.403983:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000158429d4 x1846777720762624/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228740.404033:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720762624:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228740.405390:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720762560:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228740.405484:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228740.405489:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228740.409923:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720762560:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228740.409942:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720762496:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228740.409951:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720762432:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228740.410065:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228740.410071:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228740.410075:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000158429d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720762624:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228740.464316:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228740.977224:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228741.488281:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228742.000354:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228742.511994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228743.024132:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228743.535983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228744.048124:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228744.559944:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228745.072373:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228745.520097:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228745.520104:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228745.520124:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228745.520135:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4338 00000100:00080000:0.0:1761228745.520139:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1761228745.520146:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228745.520157:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228745.520172:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000024af3661 x1846777720762880/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228745.520197:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228745.520200:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228745.520285:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720762880:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228745.520592:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720762688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228745.520649:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720762816:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228745.520673:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228745.520678:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720762752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228745.539108:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720762752:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228745.539132:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720762816:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228745.542750:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228745.542763:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228745.542772:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720762880:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228745.543413:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720762688:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228745.583949:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228746.096126:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228746.608000:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228747.122968:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228747.633632:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228748.144095:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228748.655992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228749.168153:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228749.680024:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228750.192083:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228750.640065:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228750.640074:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228750.640079:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228750.640093:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4343 00000100:00080000:0.0:1761228750.640098:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 11s 00000100:00080000:0.0:1761228750.640105:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228750.640120:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228750.640129:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720763008:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228750.640146:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000243e8b76 x1846777720763136/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228750.640175:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228750.640179:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228750.640248:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1761228750.640258:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720762944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228750.640289:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720763072:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228750.641768:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720763136:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228750.645106:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720763072:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228750.646242:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720762944:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228750.646262:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720763008:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228750.646370:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228750.646375:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228750.646379:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000243e8b76 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720763136:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228750.705598:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228751.218891:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228751.728544:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228752.240134:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228752.751966:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228753.264058:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228753.775995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228754.289031:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228754.804132:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228755.312190:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228755.760116:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720763200:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228755.760940:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228755.760950:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720763264:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228755.761696:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228755.761703:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228755.761706:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228755.761713:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4348 00000100:00080000:0.0:1761228755.761717:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1761228755.761722:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228755.761732:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228755.761744:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000024af3661 x1846777720763392/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228755.761768:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228755.761771:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228755.761834:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228755.761844:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720763328:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228755.762577:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720763392:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761228755.768085:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228755.768095:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228755.768118:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024af3661 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720763392:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228755.768221:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003e02fcda pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720763328:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228755.768254:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000805a30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720763200:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228755.769228:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035650445 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720763264:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228755.824087:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228756.336343:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228756.847974:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228757.360930:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228757.872057:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228758.384800:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228758.895984:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228759.408877:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228759.921055:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228760.432092:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228760.880230:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228760.880239:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228760.880244:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228760.880273:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4354 00000100:00080000:2.0:1761228760.880278:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228760.880286:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228760.880300:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228760.880321:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720763648/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228760.880387:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720763648:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761228760.880818:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228760.880824:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228760.880999:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720763456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228760.881070:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720763584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228760.881104:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1761228760.881109:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720763520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228760.887188:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228760.887202:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228760.887213:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720763648:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228760.887342:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720763520:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228760.887356:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720763456:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228760.887367:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720763584:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228760.944028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228761.456239:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228761.968028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228762.480904:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228762.992481:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228763.504989:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228764.016819:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228764.529304:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228765.039969:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228765.553107:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228766.001346:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720763712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228766.003236:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720763776:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228766.004410:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228766.004418:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228766.004422:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228766.004432:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4359 00000100:00080000:0.0:1761228766.004436:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228766.004442:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228766.004457:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228766.004477:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a99a8bd5 x1846777720763904/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228766.004502:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228766.004507:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228766.004550:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720763840:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228766.004715:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720763904:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228766.004911:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092ccaebf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720763712:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228766.008946:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d88fcfe6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720763840:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228766.009013:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ad2be88 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720763776:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228766.017518:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228766.017531:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228766.017542:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a99a8bd5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720763904:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228766.064030:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228766.576387:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228767.091842:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228767.600146:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228768.111994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228768.624608:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228769.137405:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228769.650757:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228770.160228:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228770.672128:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228771.122401:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228771.122409:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228771.122413:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228771.122426:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4364 00000100:00080000:3.0:1761228771.122430:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228771.122437:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228771.122451:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228771.122470:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000035883416 x1846777720764160/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1761228771.122523:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720764160:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:3.0:1761228771.122704:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228771.122708:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228771.122738:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720764032:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228771.122822:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720763968:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228771.123882:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720764096:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228771.130798:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720764032:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228771.131513:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720763968:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228771.131531:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720764096:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228771.131642:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228771.131648:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228771.131653:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720764160:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228771.184042:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228771.697760:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228772.208480:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228772.720135:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228773.232116:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228773.750532:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228774.255972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228774.768100:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228775.280036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228775.792168:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228776.240170:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720764224:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228776.240357:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228776.240362:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228776.240365:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228776.240387:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4369 00000100:00080000:3.0:1761228776.240390:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228776.240397:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228776.240407:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228776.240422:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000074fca4bf x1846777720764416/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228776.240496:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720764416:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228776.241047:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720764352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228776.241213:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228776.241217:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228776.241253:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720764288:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228776.247694:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720764352:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228776.247694:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720764288:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228776.248088:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720764224:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228776.248167:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228776.248171:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228776.248175:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720764416:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228776.306247:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228776.817229:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228777.328014:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228777.840069:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228778.353532:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228778.864090:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228779.375986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228779.888636:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228780.400002:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228780.912164:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228781.360248:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228781.360257:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228781.360261:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228781.360272:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4374 00000100:00080000:2.0:1761228781.360275:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228781.360283:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228781.360294:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228781.360311:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006be78503 x1846777720764672/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228781.360338:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228781.360342:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228781.360368:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720764544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228781.360467:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1761228781.360489:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720764480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228781.360514:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720764608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228781.361043:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720764672:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228781.366132:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720764480:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228781.371419:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228781.371431:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228781.371441:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be78503 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720764672:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228781.371528:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720764608:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228781.371542:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720764544:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228781.424681:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228781.936444:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228782.451889:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228782.960444:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228783.471972:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228783.984495:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228784.495983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228785.008946:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228785.519998:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228786.032040:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228786.480129:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228786.480137:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228786.480142:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228786.480153:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4379 00000100:00080000:3.0:1761228786.480157:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228786.480179:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228786.480189:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228786.480205:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720764928/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228786.480231:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228786.480235:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:3.0:1761228786.480259:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720764800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228786.480348:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1761228786.480353:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720764736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228786.480371:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720764864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228786.480490:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720764928:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228786.487113:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720764736:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228786.487135:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720764864:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228786.487144:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720764800:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228786.489352:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228786.489364:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228786.489373:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720764928:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228786.544012:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228787.057120:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228787.568251:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228788.080471:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228788.592290:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228789.107242:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228789.617277:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228790.130130:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228790.640006:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228791.153047:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228791.600509:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720764992:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228791.601357:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228791.601369:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720765056:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228791.602002:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228791.602016:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228791.602021:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228791.602030:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4384 00000100:00080000:0.0:1761228791.602034:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228791.602041:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228791.602053:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228791.602073:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d991e6ad x1846777720765184/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761228791.602154:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720765184:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228791.602248:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720765120:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228791.603390:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228791.603401:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228791.607511:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720765056:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228791.608296:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228791.608304:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228791.608311:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d991e6ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720765184:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228791.608633:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720765120:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228791.608644:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720764992:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228791.664534:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228792.176915:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228792.688232:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228793.200917:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228793.712019:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228794.230091:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228794.737370:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228795.248848:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228795.760888:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228796.272044:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228796.720750:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720765248:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228796.720755:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228796.720762:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228796.720766:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228796.720772:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4389 00000100:00080000:2.0:1761228796.720775:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228796.720782:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228796.720795:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228796.720812:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000f48001e x1846777720765440/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228796.720842:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228796.720846:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228796.720889:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720765312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228796.720891:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720765376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228796.720941:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720765440:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:0.0:1761228796.728570:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1761228796.728577:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3214ada pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720765312:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228796.728581:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228796.728586:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000f48001e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720765440:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228796.728681:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720765376:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228796.728694:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720765248:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228796.784123:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228797.296245:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228797.809468:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228798.320122:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228798.832034:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228799.344185:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228799.855987:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228800.369048:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228800.884078:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228801.392214:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1761228801.841687:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228801.841695:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:1.0:1761228801.841699:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1761228801.841711:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4394 00000100:00100000:0.0:1761228801.841712:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720765504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:1.0:1761228801.841715:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1761228801.841739:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:1.0:1761228801.841764:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:1.0:1761228801.841783:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dc5f36b1 x1846777720765696/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1761228801.841803:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1761228801.841806:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228801.841808:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720765632:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228801.841866:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720765568:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228801.842034:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720765696:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228801.854349:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a2fc13a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720765568:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228801.856358:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abc685ce pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720765504:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228801.856392:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5268d0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720765632:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228801.856670:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228801.856676:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228801.856680:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc5f36b1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720765696:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228801.904011:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228802.416762:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228802.927991:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228803.440622:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228803.952093:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228804.465592:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228804.976039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228805.488121:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228806.000167:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228806.512809:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228806.961809:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228806.961818:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228806.961822:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228806.961836:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4400 00000100:00080000:2.0:1761228806.961840:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228806.961848:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228806.961921:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228806.961942:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065587ea1 x1846777720765952/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228806.961959:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228806.961963:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228806.961999:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720765824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228806.962135:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761228806.962144:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720765760:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228806.962177:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720765888:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228806.962343:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720765952:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228806.973131:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720765824:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228806.978326:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228806.978339:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228806.978350:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065587ea1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720765952:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228806.979123:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720765888:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228806.979174:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720765760:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228807.024158:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228807.537107:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228808.048520:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228808.560714:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228809.073033:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228809.585106:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228810.096968:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228810.609274:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228811.120846:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228811.632936:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228812.080138:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228812.080146:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228812.080150:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228812.080160:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4405 00000100:00100000:2.0:1761228812.080161:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720766016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228812.080181:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228812.080191:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228812.080207:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228812.080228:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000784a1a9f x1846777720766208/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228812.080255:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228812.080260:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228812.080282:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720766144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228812.080287:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720766080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228812.080405:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720766208:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:2.0:1761228812.089272:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228812.089292:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228812.089305:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000784a1a9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720766208:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228812.091905:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b7677b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720766016:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228812.094406:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003544dbb3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720766144:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228812.094564:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720766080:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228812.148457:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228812.656382:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228813.168135:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228813.680838:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228814.191950:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228814.705315:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228815.216503:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228815.728444:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228816.240105:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228816.752233:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228817.201949:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228817.201960:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228817.201964:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228817.201977:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4410 00000100:00080000:3.0:1761228817.201981:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228817.201988:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228817.202001:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228817.202018:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000088958979 x1846777720766464/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1761228817.202047:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228817.202050:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228817.202173:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720766464:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228817.206907:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720766272:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228817.207048:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720766400:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228817.207094:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228817.207101:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720766336:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228817.221627:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000074fca4bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720766336:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228817.221645:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000727a12f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720766272:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228817.221655:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008536b8df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720766400:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228817.221825:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228817.221830:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228817.221834:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000088958979 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720766464:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228817.265469:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228817.778112:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228818.290900:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228818.800562:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228819.311963:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228819.824982:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228820.336129:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228820.848151:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228821.359986:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228821.872147:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228822.320063:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228822.320071:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228822.320075:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228822.320086:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4415 00000100:00080000:3.0:1761228822.320089:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228822.320097:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228822.320108:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228822.320126:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbd439 x1846777720766720/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228822.320139:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720766528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228822.320159:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228822.320173:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228822.320232:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720766720:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228822.320235:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720766592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228822.320274:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720766656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:0.0:1761228822.324266:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1761228822.324278:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:0.0:1761228822.324290:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbd439 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720766720:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:3.0:1761228822.324630:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720766592:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228822.324808:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035883416 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720766528:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228822.324822:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006d85789 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720766656:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228822.384031:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228822.896881:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228823.409845:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228823.920140:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228824.432935:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228824.946307:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228825.456035:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228825.969092:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228826.480641:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228826.992230:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228827.441625:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720766784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228827.442820:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228827.442828:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228827.442832:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228827.442841:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4420 00000100:00080000:0.0:1761228827.442844:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228827.442851:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228827.442882:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228827.442919:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d02a515f x1846777720766976/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1761228827.442986:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720766976:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:1.0:1761228827.443764:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720766912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228827.443814:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1761228827.443821:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720766848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:0.0:1761228827.444889:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228827.444898:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:1.0:1761228827.448039:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000085a527fc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720766912:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228827.449772:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228827.449785:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228827.449794:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d02a515f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720766976:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:0.0:1761228827.450201:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720766848:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228827.450221:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720766784:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228827.503993:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228828.017783:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228828.528933:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228829.040833:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228829.552046:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228830.064570:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228830.576184:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228831.096129:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228831.600022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228832.112729:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228832.563002:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228832.563012:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228832.563015:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228832.563029:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4425 00000100:00080000:3.0:1761228832.563033:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228832.563040:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228832.563053:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228832.563072:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d56d40dd x1846777720767232/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228832.563169:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720767232:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228832.565665:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720767040:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228832.565779:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720767168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228832.565811:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228832.565816:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720767104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228832.568213:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228832.568226:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:2.0:1761228832.571370:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228832.571384:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228832.571397:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d56d40dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720767232:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:2.0:1761228832.574592:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cd946368 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720767104:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228832.574614:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000341ad5cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720767040:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228832.574626:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b01aecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720767168:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228832.624072:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228833.137270:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228833.650165:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228834.160658:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1761228834.672898:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228835.185847:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228835.697626:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228836.209180:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228836.723576:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228837.232390:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228837.680105:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228837.680113:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228837.680117:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228837.680130:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4430 00000100:00080000:3.0:1761228837.680134:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228837.680142:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228837.680154:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228837.680172:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008dec2428 x1846777720767488/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228837.680273:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720767488:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:3.0:1761228837.680412:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228837.680416:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228837.680509:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720767296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228837.680545:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228837.680574:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1761228837.680579:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228837.686605:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003d27cd56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720767296:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228837.689716:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228837.689729:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228837.689739:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008dec2428 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720767488:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228837.689850:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9712cd6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767360:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228837.689881:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f64df33e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767424:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228837.747983:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228838.256148:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228838.767992:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228839.280109:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228839.791994:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228840.305663:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228840.815997:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228841.328145:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228841.839995:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228842.355747:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228842.800211:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228842.800221:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228842.800225:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1761228842.800236:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4435 00000100:00080000:2.0:1761228842.800240:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228842.800247:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228842.800258:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228842.800274:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046c2349c x1846777720767744/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228842.800298:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228842.800301:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228842.800323:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228842.800425:0:2287:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1761228842.800432:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720767552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228842.800475:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228842.800779:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720767744:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00080000:1.0:1761228842.803179:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228842.803191:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228842.803200:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046c2349c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720767744:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228842.805138:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720767552:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228842.805158:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767680:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228842.805169:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767616:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228842.863975:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228843.377115:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228843.888716:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228844.400119:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1761228844.912252:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228845.424155:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228845.939679:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228846.448409:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228846.961498:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228847.483963:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1761228847.920153:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228847.920163:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:2.0:1761228847.920168:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1761228847.920177:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720767808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:2.0:1761228847.920178:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4441 00000100:00080000:2.0:1761228847.920185:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1761228847.920194:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:2.0:1761228847.920209:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:2.0:1761228847.920229:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072dbf25b x1846777720768000/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1761228847.920256:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1761228847.920260:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:2.0:1761228847.920292:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720767872:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1761228847.920517:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720768000:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:3.0:1761228847.922669:0:2287:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:1.0:1761228847.938813:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228847.938830:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228847.938844:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072dbf25b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720768000:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228847.939007:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000290da3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720767872:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228847.939095:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f5134a53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2287:1846777720767808:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1761228847.939110:0:2287:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008451bff4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2287:1846777720767936:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228847.984029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228848.496148:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228849.008038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228849.520098:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228850.032024:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228850.545264:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228851.056036:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228851.569734:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228852.080022:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228852.600797:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1761228853.040097:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228853.040104:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:3.0:1761228853.040108:0:37:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1761228853.040118:0:37:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4446 00000100:00080000:3.0:1761228853.040121:0:37:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1761228853.040127:0:37:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:3.0:1761228853.040137:0:37:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:3.0:1761228853.040151:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009d0a2521 x1846777720768256/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1761228853.040214:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720768256:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228853.040392:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720768064:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228853.040431:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720768192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228853.040487:0:2288:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1761228853.040492:0:2288:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720768128:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00080000:3.0:1761228853.042522:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1761228853.042533:0:37:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00080000:1.0:1761228853.045668:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1761228853.045683:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:1.0:1761228853.045696:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d0a2521 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720768256:192.168.202.114@tcp:8:kworker/u8:2.0 00000100:00100000:2.0:1761228853.054505:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6d0bfc0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720768128:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228853.054540:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8803538 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2288:1846777720768192:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1761228853.059258:0:2288:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb9f4e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_01_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2288:1846777720768064:192.168.202.114@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1761228853.104052:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228853.617094:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228854.128049:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228854.640343:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228855.152223:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228855.666060:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228856.176017:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228856.695790:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228857.200027:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228857.713152:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228858.164330:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228858.164338:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228858.164348:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228858.164360:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4451 00000100:00080000:0.0:1761228858.164364:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228858.164371:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228858.164384:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228858.164406:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c83208a x1846777720768512/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228858.164433:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228858.164437:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228858.164495:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768384:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228858.164596:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761228858.164602:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720768320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228858.164619:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768448:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1761228858.164837:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720768512:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228858.175197:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de75897f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768384:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228858.176784:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7988e14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720768320:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228858.177383:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000129fdcbe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768448:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:2.0:1761228858.179421:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1761228858.179437:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:2.0:1761228858.179449:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c83208a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720768512:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228858.224279:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228858.736801:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228859.248028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228859.761011:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228860.272056:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228860.784087:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228861.296028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228861.809341:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228862.321389:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228862.832180:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228863.280115:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228863.280124:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228863.280129:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228863.280141:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4456 00000100:00080000:0.0:1761228863.280164:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228863.280171:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000100:00100000:1.0:1761228863.280174:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720768576:192.168.202.114@tcp:400:kworker/u8:3.0 00000008:00100000:0.0:1761228863.280183:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228863.280202:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000052859013 x1846777720768768/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228863.280231:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228863.280234:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228863.280278:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720768640:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228863.280278:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1761228863.288791:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720768768:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:1.0:1761228863.292088:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002168af17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720768576:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228863.292120:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fe9fa97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768704:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228863.292274:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720768640:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228863.292505:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228863.292515:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228863.292539:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052859013 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720768768:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228863.345050:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228863.857552:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228864.368029:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228864.880622:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228865.394207:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228865.904137:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1761228866.415980:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228866.929663:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228867.440041:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228867.952342:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228868.400138:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228868.400146:0:21340:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228868.400150:0:21340:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228868.400162:0:21340:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4461 00000100:00080000:0.0:1761228868.400166:0:21340:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228868.400173:0:21340:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228868.400186:0:21340:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228868.400205:0:21340:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720769024/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1761228868.400284:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720769024:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00080000:0.0:1761228868.400522:0:21340:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228868.400542:0:21340:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228868.400569:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228868.400612:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1761228868.400617:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720768832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228868.400641:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768960:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00080000:3.0:1761228868.406605:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228868.406619:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228868.406630:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720769024:192.168.202.114@tcp:8:kworker/u8:3.0 00000100:00100000:0.0:1761228868.408962:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2285:1846777720768832:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1761228868.408990:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768896:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228868.414264:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720768960:192.168.202.114@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1761228868.464032:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228868.979095:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228869.488026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228870.000351:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228870.516579:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228871.024968:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228871.537079:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228872.048323:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228872.561309:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228873.074947:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1761228873.521774:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228873.521783:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228873.521787:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228873.521800:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4466 00000100:00080000:0.0:1761228873.521804:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228873.521812:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228873.521825:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228873.521846:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000e956e16 x1846777720769280/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228873.521902:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228873.521906:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228873.521941:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720769152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761228873.522082:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1761228873.522089:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720769088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761228873.522113:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720769216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761228873.523142:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720769280:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761228873.532359:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720769088:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761228873.532387:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720769152:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761228873.538429:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720769216:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:02020000:3.0:1761228873.540388:0:2284:0:(client.c:1361:ptlrpc_check_status()) 11-0: lustre-OST0000-osc-ffff98ca898c0800: operation ost_connect to node 192.168.202.114@tcp failed: rc = -19 00000100:00080000:3.0:1761228873.550017:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228873.550032:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228873.550048:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e956e16 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720769280:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761228873.584038:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228874.096121:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228874.608026:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228875.120091:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228875.632039:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228876.144181:0:11335:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228876.656100:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:1.0:1761228877.168092:0:5733:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228877.680028:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds 00010000:00010000:0.0:1761228878.193500:0:5732:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1761228878.640750:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720769344:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761228878.641630:0:2286:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1761228878.641642:0:2286:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720769408:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:0.0:1761228878.642087:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228878.642094:0:14820:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-OST0000_UUID: starting recovery 00000100:00080000:0.0:1761228878.642098:0:14820:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1761228878.642105:0:14820:0:(import.c:534:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: connect to NID 192.168.202.114@tcp last attempt 4471 00000100:00080000:0.0:1761228878.642108:0:14820:0:(import.c:581:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1761228878.642115:0:14820:0:(import.c:615:import_select_connection()) lustre-OST0000-osc-ffff98ca898c0800: import 00000000d1cb207c using connection 192.168.202.114@tcp/192.168.202.114@tcp 00000008:00100000:0.0:1761228878.642128:0:14820:0:(osc_request.c:3485:osc_reconnect()) ocd_connect_flags: 0xa0465af2e3640078 ocd_version: 34539264 ocd_grant: 54525952, lost: 0. 00000100:00100000:0.0:1761228878.642146:0:14820:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a9d8669 x1846777720769536/t0(0) o8->lustre-OST0000-osc-ffff98ca898c0800@192.168.202.114@tcp:28/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1761228878.642171:0:14820:0:(pinger.c:251:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: level IDLE/11 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1761228878.642175:0:14820:0:(pinger.c:265:ptlrpc_pinger_process_import()) 774c7a2d-e1c6-4d57-85f8-be532a26f7be->lustre-OST0001_UUID: not pinging (in recovery or recovery disabled: IDLE) 00000100:00100000:0.0:1761228878.642232:0:2285:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1761228878.642243:0:2285:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720769472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:3.0:1761228878.642510:0:2284:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720769536:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:1.0:1761228878.649777:0:2285:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018849446 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2285:1846777720769472:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00080000:3.0:1761228878.649947:0:2284:0:(import.c:86:import_set_state_nolock()) 00000000d1cb207c lustre-OST0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1761228878.649959:0:2284:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-OST0000_UUID on 192.168.202.114@tcp failed (-19) 00000100:00100000:3.0:1761228878.649967:0:2284:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a9d8669 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2284:1846777720769536:192.168.202.114@tcp:8:kworker/u8:0.0 00000100:00100000:0.0:1761228878.650168:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000525c9db8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:895ac03d-5e0b-4331-b6a0-b29d8c52c12b:2286:1846777720769344:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:0.0:1761228878.650194:0:2286:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0db11ae pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:774c7a2d-e1c6-4d57-85f8-be532a26f7be:2286:1846777720769408:192.168.202.114@tcp:400:kworker/u8:0.0 00000100:00100000:1.0:1761228878.704098:0:21065:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a759fa26 going to sleep for 0 seconds Debug log: 22179 lines, 22179 kept, 0 dropped, 0 bad.